=== events in opendatahub (2026-11-02T23:01:20Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 25m Normal Scheduled pod/maas-controller-546bfbd88f-9cdcm Successfully assigned opendatahub/maas-controller-546bfbd88f-9cdcm to ip-10-0-143-75.us-west-2.compute.internal 49m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 to ip-10-0-140-221.us-west-2.compute.internal 50m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-p2rtg Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-p2rtg to ip-10-0-143-75.us-west-2.compute.internal 49m Normal Scheduled pod/model-serving-api-75bd89fdfc-8tqw4 Successfully assigned opendatahub/model-serving-api-75bd89fdfc-8tqw4 to ip-10-0-143-75.us-west-2.compute.internal 25m Normal Scheduled pod/maas-controller-546bfbd88f-2r6j4 Successfully assigned opendatahub/maas-controller-546bfbd88f-2r6j4 to ip-10-0-143-75.us-west-2.compute.internal 49m Normal Scheduled pod/kserve-controller-manager-776cc66d47-hbxxd Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-hbxxd to ip-10-0-143-75.us-west-2.compute.internal 49m Normal Scheduled pod/maas-controller-579cf7c6d-p7mck Successfully assigned opendatahub/maas-controller-579cf7c6d-p7mck to ip-10-0-130-41.us-west-2.compute.internal 46m Normal Scheduled pod/ai-gateway-controller-86c7698f44-dhr65 Successfully assigned opendatahub/ai-gateway-controller-86c7698f44-dhr65 to ip-10-0-140-221.us-west-2.compute.internal 45m Normal Scheduled pod/ai-gateway-controller-54d8d5fc5c-6k894 Successfully assigned opendatahub/ai-gateway-controller-54d8d5fc5c-6k894 to ip-10-0-140-221.us-west-2.compute.internal 49m Normal Scheduled pod/odh-model-controller-769875db56-mn4sc Successfully assigned opendatahub/odh-model-controller-769875db56-mn4sc to ip-10-0-140-221.us-west-2.compute.internal 49m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-4qbjf to ip-10-0-130-41.us-west-2.compute.internal 47m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-pjd76 Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-pjd76 to ip-10-0-130-41.us-west-2.compute.internal 34m Normal Scheduled pod/test-curl-7216-773e6d Successfully assigned opendatahub/test-curl-7216-773e6d to ip-10-0-143-75.us-west-2.compute.internal 33m Normal Scheduled pod/test-curl-7216-49cac6 Successfully assigned opendatahub/test-curl-7216-49cac6 to ip-10-0-130-41.us-west-2.compute.internal 49m Normal Scheduled pod/maas-controller-546bfbd88f-nd9wf Successfully assigned opendatahub/maas-controller-546bfbd88f-nd9wf to ip-10-0-143-75.us-west-2.compute.internal 50m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 50m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-p2rtg 50m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-p2rtg Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 50m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-p2rtg Add eth0 [10.133.0.23/23] from ovn-kubernetes 50m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-p2rtg Container started 50m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-p2rtg Container created 50m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-p2rtg Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.068s (3.068s including waiting). Image size: 195729000 bytes. 49m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-p2rtg_53394c84-befa-4aaa-8561-f817f1cff39e became leader 49m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-579cf7c6d from 0 to 1 49m Normal SuccessfulCreate replicaset/maas-controller-579cf7c6d Created pod: maas-controller-579cf7c6d-p7mck 49m Normal Pulling pod/maas-controller-579cf7c6d-p7mck Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 49m Normal AddedInterface pod/maas-controller-579cf7c6d-p7mck Add eth0 [10.132.0.40/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/maas-controller-546bfbd88f Created pod: maas-controller-546bfbd88f-nd9wf 49m Normal Pulling pod/maas-controller-546bfbd88f-nd9wf Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 49m Normal AddedInterface pod/maas-controller-546bfbd88f-nd9wf Add eth0 [10.133.0.25/23] from ovn-kubernetes 49m Normal Started pod/maas-controller-579cf7c6d-p7mck Container started 49m Normal Created pod/maas-controller-579cf7c6d-p7mck Container created 49m Normal Pulled pod/maas-controller-579cf7c6d-p7mck Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.185s (3.185s including waiting). Image size: 200053271 bytes. 49m Normal Pulled pod/maas-controller-546bfbd88f-nd9wf Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.166s (3.166s including waiting). Image size: 200053271 bytes. 49m Normal Created pod/maas-controller-546bfbd88f-nd9wf Container created 49m Normal Started pod/maas-controller-546bfbd88f-nd9wf Container started 49m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-579cf7c6d-p7mck_218f6448-84cf-4457-961c-2d24c795830a became leader 49m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 49m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-kqst9 49m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 49m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 49m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 49m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 Add eth0 [10.134.0.40/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-579cf7c6d from 1 to 0 49m Normal SuccessfulDelete replicaset/maas-controller-579cf7c6d Deleted pod: maas-controller-579cf7c6d-p7mck 49m Normal Killing pod/maas-controller-579cf7c6d-p7mck Stopping container manager 49m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 12.812s (12.812s including waiting). Image size: 1089502687 bytes. 49m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 Container created 49m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-kqst9 Container started 49m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-546bfbd88f-nd9wf_6b447569-5158-4766-a6ec-67a09394ff2e became leader 49m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 49m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 49m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Add eth0 [10.132.0.42/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 49m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-mn4sc 49m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-4qbjf 49m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-hbxxd 49m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 49m Normal AddedInterface pod/model-serving-api-75bd89fdfc-8tqw4 Add eth0 [10.133.0.28/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 49m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-8tqw4 49m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-hbxxd Add eth0 [10.133.0.27/23] from ovn-kubernetes 49m Normal Pulling pod/kserve-controller-manager-776cc66d47-hbxxd Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 49m Normal Pulling pod/model-serving-api-75bd89fdfc-8tqw4 Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 49m Normal AddedInterface pod/odh-model-controller-769875db56-mn4sc Add eth0 [10.134.0.41/23] from ovn-kubernetes 49m Normal Pulling pod/odh-model-controller-769875db56-mn4sc Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 49m Normal Pulled pod/model-serving-api-75bd89fdfc-8tqw4 Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.56s (2.56s including waiting). Image size: 184925761 bytes. 49m Normal Started pod/model-serving-api-75bd89fdfc-8tqw4 Container started 49m Normal Started pod/kserve-controller-manager-776cc66d47-hbxxd Container started 49m Normal Created pod/kserve-controller-manager-776cc66d47-hbxxd Container created 49m Normal Pulled pod/kserve-controller-manager-776cc66d47-hbxxd Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.769s (2.769s including waiting). Image size: 247676581 bytes. 49m Normal Created pod/model-serving-api-75bd89fdfc-8tqw4 Container created 49m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-hbxxd_e8054af7-db2a-4d54-adfe-a1a0e18cee39 became leader 49m Normal Created pod/odh-model-controller-769875db56-mn4sc Container created 49m Normal Started pod/odh-model-controller-769875db56-mn4sc Container started 49m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Container created 49m Normal Pulled pod/odh-model-controller-769875db56-mn4sc Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.425s (3.425s including waiting). Image size: 236755567 bytes. 49m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Container started 49m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.198s (3.198s including waiting). Image size: 235100981 bytes. 49m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-mn4sc_9d006adc-8d76-4cec-93e5-878a9684f23e became leader 48m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-4qbjf_b94bc41c-c03c-440d-9629-41596edd319c became leader 47m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-pjd76 47m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 47m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-pjd76 Add eth0 [10.132.0.45/23] from ovn-kubernetes 47m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-pjd76 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 47m Normal Started pod/llmisvc-controller-manager-6466c554d8-pjd76 Container started 47m Normal Created pod/llmisvc-controller-manager-6466c554d8-pjd76 Container created 47m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-pjd76 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 919ms (919ms including waiting). Image size: 235100981 bytes. 47m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-4qbjf 47m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-4qbjf Stopping container manager 47m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 47m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-pjd76_ec71b9c5-51aa-4ec0-a396-e7edd6fde6c8 became leader 46m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-86c7698f44 from 0 to 1 46m Normal SuccessfulCreate replicaset/ai-gateway-controller-86c7698f44 Created pod: ai-gateway-controller-86c7698f44-dhr65 46m Normal AddedInterface pod/ai-gateway-controller-86c7698f44-dhr65 Add eth0 [10.134.0.49/23] from ovn-kubernetes 46m Normal Pulling pod/ai-gateway-controller-86c7698f44-dhr65 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-211" 45m Normal Pulled pod/ai-gateway-controller-86c7698f44-dhr65 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-211" in 7.611s (7.611s including waiting). Image size: 183998371 bytes. 45m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-86c7698f44-dhr65_6838e20b-738d-4afb-8890-b437f6c15b1b became leader 45m Normal Created pod/ai-gateway-controller-86c7698f44-dhr65 Container created 45m Normal Started pod/ai-gateway-controller-86c7698f44-dhr65 Container started 45m Normal SuccessfulCreate replicaset/ai-gateway-controller-54d8d5fc5c Created pod: ai-gateway-controller-54d8d5fc5c-6k894 45m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-54d8d5fc5c from 0 to 1 45m Normal Pulling pod/ai-gateway-controller-54d8d5fc5c-6k894 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-211" 45m Normal AddedInterface pod/ai-gateway-controller-54d8d5fc5c-6k894 Add eth0 [10.134.0.51/23] from ovn-kubernetes 45m Normal Started pod/ai-gateway-controller-54d8d5fc5c-6k894 Container started 45m Normal Created pod/ai-gateway-controller-54d8d5fc5c-6k894 Container created 45m Normal Pulled pod/ai-gateway-controller-54d8d5fc5c-6k894 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-211" in 908ms (908ms including waiting). Image size: 183998371 bytes. 45m Normal SuccessfulDelete replicaset/ai-gateway-controller-86c7698f44 Deleted pod: ai-gateway-controller-86c7698f44-dhr65 45m Normal Killing pod/ai-gateway-controller-86c7698f44-dhr65 Stopping container manager 45m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-86c7698f44 from 1 to 0 45m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-54d8d5fc5c-6k894_4cb39f4c-b704-4c5e-9d8a-0a5a1f895764 became leader 33m Normal AddedInterface pod/test-curl-7216-773e6d Add eth0 [10.133.0.129/23] from ovn-kubernetes 33m Normal Pulling pod/test-curl-7216-773e6d Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 33m Normal Pulled pod/test-curl-7216-773e6d Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 20.715s (20.715s including waiting). Image size: 106100986 bytes. 33m Normal Created pod/test-curl-7216-773e6d Container created 33m Normal Started pod/test-curl-7216-773e6d Container started 33m Normal Killing pod/test-curl-7216-773e6d Stopping container test-curl-7216-773e6d 33m Normal AddedInterface pod/test-curl-7216-49cac6 Add eth0 [10.132.0.88/23] from ovn-kubernetes 33m Normal Pulling pod/test-curl-7216-49cac6 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 33m Normal Pulled pod/test-curl-7216-49cac6 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.899s (2.899s including waiting). Image size: 106100986 bytes. 33m Normal Created pod/test-curl-7216-49cac6 Container created 33m Normal Started pod/test-curl-7216-49cac6 Container started 33m Normal Killing pod/test-curl-7216-49cac6 Stopping container test-curl-7216-49cac6 25m Normal SuccessfulDelete replicaset/maas-controller-546bfbd88f Deleted pod: maas-controller-546bfbd88f-nd9wf 25m Normal Killing pod/maas-controller-546bfbd88f-nd9wf Stopping container manager 25m Normal Pulling pod/maas-controller-546bfbd88f-2r6j4 Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 25m Normal SuccessfulCreate replicaset/maas-controller-546bfbd88f Created pod: maas-controller-546bfbd88f-2r6j4 25m Normal AddedInterface pod/maas-controller-546bfbd88f-2r6j4 Add eth0 [10.133.0.186/23] from ovn-kubernetes 25m Normal Started pod/maas-controller-546bfbd88f-2r6j4 Container started 25m Normal Created pod/maas-controller-546bfbd88f-2r6j4 Container created 25m Normal Pulled pod/maas-controller-546bfbd88f-2r6j4 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 965ms (965ms including waiting). Image size: 200053271 bytes. 25m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-546bfbd88f-2r6j4_9e72fd07-17a2-4ec5-bd95-ca772e52faf8 became leader 25m Normal Killing pod/maas-controller-546bfbd88f-2r6j4 Stopping container manager 25m Normal SuccessfulDelete replicaset/maas-controller-546bfbd88f Deleted pod: maas-controller-546bfbd88f-2r6j4 25m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-546bfbd88f from 1 to 0 25m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-546bfbd88f from 0 to 1 25m Normal SuccessfulCreate replicaset/maas-controller-546bfbd88f Created pod: maas-controller-546bfbd88f-9cdcm 25m Normal AddedInterface pod/maas-controller-546bfbd88f-9cdcm Add eth0 [10.133.0.188/23] from ovn-kubernetes 25m Normal Pulling pod/maas-controller-546bfbd88f-9cdcm Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 25m Normal Started pod/maas-controller-546bfbd88f-9cdcm Container started 25m Normal Created pod/maas-controller-546bfbd88f-9cdcm Container created 25m Normal Pulled pod/maas-controller-546bfbd88f-9cdcm Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 881ms (881ms including waiting). Image size: 200053271 bytes. 24m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-546bfbd88f-9cdcm_b4e61f07-b9d0-49c4-98cb-6e576d1ab11f became leader