=== events in opendatahub (2026-11-04T04:03:55Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 6m20s Normal Scheduled pod/ai-gateway-controller-69464886d7-hpcwt Successfully assigned opendatahub/ai-gateway-controller-69464886d7-hpcwt to ip-10-0-129-253.us-east-2.compute.internal 7m43s Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-5xmx7 Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-5xmx7 to ip-10-0-129-134.us-east-2.compute.internal 3m13s Normal Scheduled pod/test-curl-6866-d6274f Successfully assigned opendatahub/test-curl-6866-d6274f to ip-10-0-129-253.us-east-2.compute.internal 3m13s Normal Scheduled pod/test-curl-6863-658bdb Successfully assigned opendatahub/test-curl-6863-658bdb to ip-10-0-138-9.us-east-2.compute.internal 8m45s Normal Scheduled pod/odh-model-controller-769875db56-xdp9r Successfully assigned opendatahub/odh-model-controller-769875db56-xdp9r to ip-10-0-138-9.us-east-2.compute.internal 8m45s Normal Scheduled pod/model-serving-api-75bd89fdfc-2m46k Successfully assigned opendatahub/model-serving-api-75bd89fdfc-2m46k to ip-10-0-138-9.us-east-2.compute.internal 8m45s Normal Scheduled pod/kserve-controller-manager-776cc66d47-58vj8 Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-58vj8 to ip-10-0-138-9.us-east-2.compute.internal 8m45s Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-xjm8c to ip-10-0-129-134.us-east-2.compute.internal 8m54s Normal Scheduled pod/maas-controller-55cc4d9668-f6wtl Successfully assigned opendatahub/maas-controller-55cc4d9668-f6wtl to ip-10-0-138-9.us-east-2.compute.internal 6m3s Normal Scheduled pod/ai-gateway-controller-79b649855b-9p994 Successfully assigned opendatahub/ai-gateway-controller-79b649855b-9p994 to ip-10-0-129-253.us-east-2.compute.internal 8m49s Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r66wn to ip-10-0-129-134.us-east-2.compute.internal 8m55s Normal Scheduled pod/maas-controller-8b95c9c66-xqc54 Successfully assigned opendatahub/maas-controller-8b95c9c66-xqc54 to ip-10-0-138-9.us-east-2.compute.internal 9m40s Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-sdgx5 Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-sdgx5 to ip-10-0-138-9.us-east-2.compute.internal 9m41s Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-sdgx5 9m41s Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 9m41s Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-sdgx5 Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 9m41s Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-sdgx5 Add eth0 [10.132.0.43/23] from ovn-kubernetes 9m38s Normal Started pod/kserve-module-controller-manager-7566cf5fc8-sdgx5 Container started 9m38s Normal Created pod/kserve-module-controller-manager-7566cf5fc8-sdgx5 Container created 9m38s Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-sdgx5 Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.846s (2.846s including waiting). Image size: 195729000 bytes. 9m21s Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-sdgx5_6a4b728a-a98e-443b-a0c7-23c3f21a37cc became leader 8m56s Normal SuccessfulCreate replicaset/maas-controller-8b95c9c66 Created pod: maas-controller-8b95c9c66-xqc54 8m56s Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-8b95c9c66 from 0 to 1 8m55s Normal AddedInterface pod/maas-controller-55cc4d9668-f6wtl Add eth0 [10.132.0.45/23] from ovn-kubernetes 8m55s Normal AddedInterface pod/maas-controller-8b95c9c66-xqc54 Add eth0 [10.132.0.44/23] from ovn-kubernetes 8m55s Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-55cc4d9668 from 0 to 1 8m55s Normal Pulling pod/maas-controller-55cc4d9668-f6wtl Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 8m55s Normal Pulling pod/maas-controller-8b95c9c66-xqc54 Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 8m55s Normal SuccessfulCreate replicaset/maas-controller-55cc4d9668 Created pod: maas-controller-55cc4d9668-f6wtl 8m52s Normal Started pod/maas-controller-55cc4d9668-f6wtl Container started 8m52s Normal Created pod/maas-controller-55cc4d9668-f6wtl Container created 8m52s Normal Pulled pod/maas-controller-55cc4d9668-f6wtl Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 3.121s (3.121s including waiting). Image size: 200103442 bytes. 8m51s Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-55cc4d9668-f6wtl_0ebc20f2-11f8-4b0b-9148-4f591237b7f0 became leader 8m51s Normal Pulled pod/maas-controller-8b95c9c66-xqc54 Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 3.976s (3.976s including waiting). Image size: 200103442 bytes. 8m51s Normal Created pod/maas-controller-8b95c9c66-xqc54 Container created 8m51s Normal Started pod/maas-controller-8b95c9c66-xqc54 Container started 8m50s Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 8m50s Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-r66wn 8m50s Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 8m49s Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 8m49s Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn Add eth0 [10.133.0.33/23] from ovn-kubernetes 8m46s Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-xdp9r 8m46s Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-58vj8 8m46s Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 8m46s Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-2m46k 8m46s Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 8m46s Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 8m46s Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-xjm8c 8m46s Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 8m45s Normal Pulling pod/kserve-controller-manager-776cc66d47-58vj8 Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 8m45s Normal Pulling pod/odh-model-controller-769875db56-xdp9r Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 8m45s Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Add eth0 [10.133.0.34/23] from ovn-kubernetes 8m45s Normal AddedInterface pod/model-serving-api-75bd89fdfc-2m46k Add eth0 [10.132.0.47/23] from ovn-kubernetes 8m45s Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 8m45s Normal Pulling pod/model-serving-api-75bd89fdfc-2m46k Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 8m45s Normal AddedInterface pod/odh-model-controller-769875db56-xdp9r Add eth0 [10.132.0.48/23] from ovn-kubernetes 8m45s Normal AddedInterface pod/kserve-controller-manager-776cc66d47-58vj8 Add eth0 [10.132.0.46/23] from ovn-kubernetes 8m42s Normal Pulled pod/kserve-controller-manager-776cc66d47-58vj8 Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.11s (3.11s including waiting). Image size: 247676581 bytes. 8m42s Normal Created pod/model-serving-api-75bd89fdfc-2m46k Container created 8m42s Normal Started pod/model-serving-api-75bd89fdfc-2m46k Container started 8m42s Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-58vj8_0ad45101-b9ea-4f61-a9ad-7cda045a8f7a became leader 8m42s Normal Pulled pod/model-serving-api-75bd89fdfc-2m46k Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.741s (2.741s including waiting). Image size: 184925761 bytes. 8m42s Normal Started pod/kserve-controller-manager-776cc66d47-58vj8 Container started 8m42s Normal Created pod/kserve-controller-manager-776cc66d47-58vj8 Container created 8m41s Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-xdp9r_1adf1c37-826b-4ad8-89b1-4fe47335bcf4 became leader 8m41s Normal Created pod/odh-model-controller-769875db56-xdp9r Container created 8m41s Normal Started pod/odh-model-controller-769875db56-xdp9r Container started 8m41s Normal Pulled pod/odh-model-controller-769875db56-xdp9r Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.519s (3.52s including waiting). Image size: 236755567 bytes. 8m41s Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.634s (3.634s including waiting). Image size: 235100981 bytes. 8m41s Normal Created pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Container created 8m41s Normal Started pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Container started 8m40s Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-xjm8c_2d64a12f-c9fe-4723-804c-9fc45619c8f7 became leader 8m40s Normal Killing pod/maas-controller-8b95c9c66-xqc54 Stopping container manager 8m40s Normal SuccessfulDelete replicaset/maas-controller-8b95c9c66 Deleted pod: maas-controller-8b95c9c66-xqc54 8m40s Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-8b95c9c66 from 1 to 0 8m35s Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.803s (13.803s including waiting). Image size: 1089502687 bytes. 8m35s Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn Container created 8m35s Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-r66wn Container started 7m44s Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-5xmx7 7m44s Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 7m44s Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-5xmx7 Add eth0 [10.133.0.38/23] from ovn-kubernetes 7m44s Normal Pulling pod/llmisvc-controller-manager-6466c554d8-5xmx7 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 7m43s Normal Pulled pod/llmisvc-controller-manager-6466c554d8-5xmx7 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 414ms (414ms including waiting). Image size: 235100981 bytes. 7m43s Normal Started pod/llmisvc-controller-manager-6466c554d8-5xmx7 Container started 7m43s Normal Created pod/llmisvc-controller-manager-6466c554d8-5xmx7 Container created 7m10s Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-xjm8c Stopping container manager 7m10s Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 7m10s Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-xjm8c 6m51s Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-5xmx7_06338e98-e4c4-447c-a7e9-7a6a96f43515 became leader 6m21s Normal SuccessfulCreate replicaset/ai-gateway-controller-69464886d7 Created pod: ai-gateway-controller-69464886d7-hpcwt 6m21s Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-69464886d7 from 0 to 1 6m20s Normal AddedInterface pod/ai-gateway-controller-69464886d7-hpcwt Add eth0 [10.134.0.33/23] from ovn-kubernetes 6m20s Normal Pulling pod/ai-gateway-controller-69464886d7-hpcwt Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" 6m16s Normal Pulled pod/ai-gateway-controller-69464886d7-hpcwt Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" in 4.403s (4.403s including waiting). Image size: 184136095 bytes. 6m16s Normal Created pod/ai-gateway-controller-69464886d7-hpcwt Container created 6m16s Normal Started pod/ai-gateway-controller-69464886d7-hpcwt Container started 6m14s Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-69464886d7-hpcwt_ca9454c5-3215-43b0-9b41-0b1b59241518 became leader 6m4s Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-79b649855b from 0 to 1 6m4s Normal SuccessfulCreate replicaset/ai-gateway-controller-79b649855b Created pod: ai-gateway-controller-79b649855b-9p994 6m3s Normal Pulling pod/ai-gateway-controller-79b649855b-9p994 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" 6m3s Normal AddedInterface pod/ai-gateway-controller-79b649855b-9p994 Add eth0 [10.134.0.35/23] from ovn-kubernetes 6m2s Normal Started pod/ai-gateway-controller-79b649855b-9p994 Container started 6m2s Normal Pulled pod/ai-gateway-controller-79b649855b-9p994 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" in 447ms (447ms including waiting). Image size: 184136095 bytes. 6m2s Normal Created pod/ai-gateway-controller-79b649855b-9p994 Container created 5m51s Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-69464886d7 from 1 to 0 5m51s Normal Killing pod/ai-gateway-controller-69464886d7-hpcwt Stopping container manager 5m51s Normal SuccessfulDelete replicaset/ai-gateway-controller-69464886d7 Deleted pod: ai-gateway-controller-69464886d7-hpcwt 5m34s Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-79b649855b-9p994_17d1c4ab-9292-482b-bb4a-d942ca59ee56 became leader 3m13s Normal AddedInterface pod/test-curl-6863-658bdb Add eth0 [10.132.0.77/23] from ovn-kubernetes 3m13s Normal Pulling pod/test-curl-6863-658bdb Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 3m13s Normal Pulling pod/test-curl-6866-d6274f Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 3m13s Normal AddedInterface pod/test-curl-6866-d6274f Add eth0 [10.134.0.56/23] from ovn-kubernetes 3m12s Normal Pulled pod/test-curl-6863-658bdb Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 1.066s (1.066s including waiting). Image size: 106100986 bytes. 3m12s Normal Started pod/test-curl-6863-658bdb Container started 3m12s Normal Created pod/test-curl-6863-658bdb Container created 3m10s Normal Started pod/test-curl-6866-d6274f Container started 3m10s Normal Created pod/test-curl-6866-d6274f Container created 3m10s Normal Pulled pod/test-curl-6866-d6274f Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.821s (2.821s including waiting). Image size: 106100986 bytes. 3m9s Normal Killing pod/test-curl-6863-658bdb Stopping container test-curl-6863-658bdb 3m8s Normal Killing pod/test-curl-6866-d6274f Stopping container test-curl-6866-d6274f