=== events in opendatahub (2026-11-04T02:26:27Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 11m Normal Scheduled pod/ai-gateway-controller-69464886d7-7lf7z Successfully assigned opendatahub/ai-gateway-controller-69464886d7-7lf7z to ip-10-0-136-63.us-west-1.compute.internal 14m Normal Scheduled pod/kserve-controller-manager-776cc66d47-jp7lh Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-jp7lh to ip-10-0-136-63.us-west-1.compute.internal 14m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 to ip-10-0-138-45.us-west-1.compute.internal 5m6s Normal Scheduled pod/test-curl-6747-849881 Successfully assigned opendatahub/test-curl-6747-849881 to ip-10-0-138-45.us-west-1.compute.internal 5m10s Normal Scheduled pod/test-curl-6747-78f208 Successfully assigned opendatahub/test-curl-6747-78f208 to ip-10-0-138-45.us-west-1.compute.internal 14m Normal Scheduled pod/odh-model-controller-769875db56-24ksh Successfully assigned opendatahub/odh-model-controller-769875db56-24ksh to ip-10-0-138-45.us-west-1.compute.internal 14m Normal Scheduled pod/model-serving-api-75bd89fdfc-52dc4 Successfully assigned opendatahub/model-serving-api-75bd89fdfc-52dc4 to ip-10-0-136-63.us-west-1.compute.internal 13m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-txbbr Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-txbbr to ip-10-0-135-225.us-west-1.compute.internal 14m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-dwnhr to ip-10-0-135-225.us-west-1.compute.internal 14m Normal Scheduled pod/maas-controller-8b95c9c66-sv5bj Successfully assigned opendatahub/maas-controller-8b95c9c66-sv5bj to ip-10-0-136-63.us-west-1.compute.internal 14m Normal Scheduled pod/maas-controller-f759c744c-cr7gd Successfully assigned opendatahub/maas-controller-f759c744c-cr7gd to ip-10-0-136-63.us-west-1.compute.internal 15m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-cwztd Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-cwztd to ip-10-0-135-225.us-west-1.compute.internal 11m Normal Scheduled pod/ai-gateway-controller-7cf899df5-cdbnv Successfully assigned opendatahub/ai-gateway-controller-7cf899df5-cdbnv to ip-10-0-136-63.us-west-1.compute.internal 15m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-cwztd 15m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 15m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-cwztd Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 15m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-cwztd Add eth0 [10.134.0.19/23] from ovn-kubernetes 15m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-cwztd Container created 15m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-cwztd Container started 15m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-cwztd Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.843s (2.843s including waiting). Image size: 195729000 bytes. 15m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-cwztd_486519bd-581d-467c-ae8e-5f97faf140dd became leader 14m Normal SuccessfulCreate replicaset/maas-controller-8b95c9c66 Created pod: maas-controller-8b95c9c66-sv5bj 14m Normal SuccessfulCreate replicaset/maas-controller-f759c744c Created pod: maas-controller-f759c744c-cr7gd 14m Warning FailedMount pod/maas-controller-8b95c9c66-sv5bj MountVolume.SetUp failed for volume "webhook-cert" : secret "maas-controller-webhook-cert" not found 14m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-8b95c9c66 from 0 to 1 14m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-f759c744c from 0 to 1 14m Normal AddedInterface pod/maas-controller-8b95c9c66-sv5bj Add eth0 [10.132.0.42/23] from ovn-kubernetes 14m Normal AddedInterface pod/maas-controller-f759c744c-cr7gd Add eth0 [10.132.0.43/23] from ovn-kubernetes 14m Normal Pulling pod/maas-controller-8b95c9c66-sv5bj Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 14m Normal Pulled pod/maas-controller-8b95c9c66-sv5bj Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 5.096s (5.096s including waiting). Image size: 200103442 bytes. 14m Normal Created pod/maas-controller-8b95c9c66-sv5bj Container created 14m Normal Started pod/maas-controller-8b95c9c66-sv5bj Container started 14m Normal Pulled pod/maas-controller-f759c744c-cr7gd Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 5.276s (5.276s including waiting). Image size: 200103442 bytes. 14m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-8b95c9c66-sv5bj_dd27c1f6-1ef4-4efd-911b-ebb60e979180 became leader 14m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 14m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 14m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 14m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 14m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 14m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 Add eth0 [10.133.0.38/23] from ovn-kubernetes 14m Normal Killing pod/maas-controller-8b95c9c66-sv5bj Stopping container manager 14m Normal SuccessfulDelete replicaset/maas-controller-8b95c9c66 Deleted pod: maas-controller-8b95c9c66-sv5bj 14m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-8b95c9c66 from 1 to 0 14m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.736s (13.736s including waiting). Image size: 1089502687 bytes. 14m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 Container started 14m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-nwmb4 Container created 14m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-jp7lh 14m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 14m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-52dc4 14m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 14m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 14m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 14m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-24ksh 14m Normal AddedInterface pod/odh-model-controller-769875db56-24ksh Add eth0 [10.133.0.39/23] from ovn-kubernetes 14m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-jp7lh Add eth0 [10.132.0.44/23] from ovn-kubernetes 14m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-dwnhr 14m Normal Pulling pod/kserve-controller-manager-776cc66d47-jp7lh Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 14m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 14m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Add eth0 [10.134.0.22/23] from ovn-kubernetes 14m Normal Pulling pod/model-serving-api-75bd89fdfc-52dc4 Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 14m Normal Pulling pod/odh-model-controller-769875db56-24ksh Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 14m Normal AddedInterface pod/model-serving-api-75bd89fdfc-52dc4 Add eth0 [10.132.0.45/23] from ovn-kubernetes 14m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-f759c744c-cr7gd_42cb955d-4df5-48b5-b3b5-b2b64213a1eb became leader 14m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.07s (3.07s including waiting). Image size: 235100981 bytes. 14m Normal Pulled pod/model-serving-api-75bd89fdfc-52dc4 Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.581s (2.581s including waiting). Image size: 184925761 bytes. 14m Normal Started pod/model-serving-api-75bd89fdfc-52dc4 Container started 14m Normal Created pod/model-serving-api-75bd89fdfc-52dc4 Container created 14m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Container started 14m Normal Pulled pod/kserve-controller-manager-776cc66d47-jp7lh Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.371s (3.371s including waiting). Image size: 247676581 bytes. 14m Normal Started pod/kserve-controller-manager-776cc66d47-jp7lh Container started 14m Normal Created pod/kserve-controller-manager-776cc66d47-jp7lh Container created 14m Normal Started pod/odh-model-controller-769875db56-24ksh Container started 14m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Container created 14m Normal Created pod/odh-model-controller-769875db56-24ksh Container created 14m Normal Pulled pod/odh-model-controller-769875db56-24ksh Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.177s (3.177s including waiting). Image size: 236755567 bytes. 14m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-dwnhr_300dd67b-5134-46a4-8266-320ff36ab58c became leader 14m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-jp7lh_b96ac047-fa3a-43fc-b705-22c5582321a4 became leader 14m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-24ksh_714563ac-0984-4ea5-a6a7-d5f1a2a06220 became leader 13m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-txbbr 13m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-txbbr Add eth0 [10.134.0.27/23] from ovn-kubernetes 13m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-txbbr Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 13m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 13m Normal Created pod/llmisvc-controller-manager-6466c554d8-txbbr Container created 13m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-txbbr Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 766ms (766ms including waiting). Image size: 235100981 bytes. 13m Normal Started pod/llmisvc-controller-manager-6466c554d8-txbbr Container started 12m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-dwnhr Stopping container manager 12m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 12m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-dwnhr 12m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-txbbr_1a1daa98-f00a-4e72-9d71-c1fc9c82fc84 became leader 12m Normal Pulling pod/maas-controller-f759c744c-cr7gd Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 12m Normal Started pod/maas-controller-f759c744c-cr7gd Container started 12m Normal Pulled pod/maas-controller-f759c744c-cr7gd Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 929ms (929ms including waiting). Image size: 200103442 bytes. 12m Normal Created pod/maas-controller-f759c744c-cr7gd Container created 11m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-f759c744c-cr7gd_f61f572a-1242-482b-9497-665383e100c7 became leader 11m Normal SuccessfulCreate replicaset/ai-gateway-controller-69464886d7 Created pod: ai-gateway-controller-69464886d7-7lf7z 11m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-69464886d7 from 0 to 1 11m Normal AddedInterface pod/ai-gateway-controller-69464886d7-7lf7z Add eth0 [10.132.0.51/23] from ovn-kubernetes 11m Normal Pulling pod/ai-gateway-controller-69464886d7-7lf7z Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" 11m Normal Started pod/ai-gateway-controller-69464886d7-7lf7z Container started 11m Normal Pulled pod/ai-gateway-controller-69464886d7-7lf7z Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" in 8.249s (8.249s including waiting). Image size: 184136094 bytes. 11m Normal Created pod/ai-gateway-controller-69464886d7-7lf7z Container created 11m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-69464886d7-7lf7z_6fe7c899-a559-464c-82f2-d0fb8de1430b became leader 11m Normal AddedInterface pod/ai-gateway-controller-7cf899df5-cdbnv Add eth0 [10.132.0.56/23] from ovn-kubernetes 11m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-7cf899df5 from 0 to 1 11m Normal SuccessfulCreate replicaset/ai-gateway-controller-7cf899df5 Created pod: ai-gateway-controller-7cf899df5-cdbnv 11m Normal Pulling pod/ai-gateway-controller-7cf899df5-cdbnv Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" 11m Normal Pulled pod/ai-gateway-controller-7cf899df5-cdbnv Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-221" in 867ms (867ms including waiting). Image size: 184136094 bytes. 11m Normal Started pod/ai-gateway-controller-7cf899df5-cdbnv Container started 11m Normal Created pod/ai-gateway-controller-7cf899df5-cdbnv Container created 10m Normal SuccessfulDelete replicaset/ai-gateway-controller-69464886d7 Deleted pod: ai-gateway-controller-69464886d7-7lf7z 10m Normal Killing pod/ai-gateway-controller-69464886d7-7lf7z Stopping container manager 10m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-69464886d7 from 1 to 0 10m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-7cf899df5-cdbnv_85fe879a-eb32-44b9-a344-6313e00494d4 became leader 5m9s Normal Pulling pod/test-curl-6747-78f208 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 5m9s Normal AddedInterface pod/test-curl-6747-78f208 Add eth0 [10.133.0.65/23] from ovn-kubernetes 5m8s Normal Created pod/test-curl-6747-78f208 Container created 5m8s Normal Started pod/test-curl-6747-78f208 Container started 5m8s Normal Pulled pod/test-curl-6747-78f208 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 1.047s (1.047s including waiting). Image size: 106100986 bytes. 5m6s Normal Started pod/test-curl-6747-849881 Container started 5m6s Normal Created pod/test-curl-6747-849881 Container created 5m6s Normal Pulled pod/test-curl-6747-849881 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 5m6s Normal AddedInterface pod/test-curl-6747-849881 Add eth0 [10.133.0.66/23] from ovn-kubernetes 5m6s Normal Killing pod/test-curl-6747-78f208 Stopping container test-curl-6747-78f208 5m3s Normal Killing pod/test-curl-6747-849881 Stopping container test-curl-6747-849881