=== events in opendatahub (2026-10-31T01:32:42Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 24m Normal Scheduled pod/ai-gateway-controller-569c46f4cf-fzxcn Successfully assigned opendatahub/ai-gateway-controller-569c46f4cf-fzxcn to ip-10-0-130-39.us-east-2.compute.internal 27m Normal Scheduled pod/maas-controller-579cf7c6d-ndnw7 Successfully assigned opendatahub/maas-controller-579cf7c6d-ndnw7 to ip-10-0-132-179.us-east-2.compute.internal 27m Normal Scheduled pod/odh-model-controller-769875db56-nmdfz Successfully assigned opendatahub/odh-model-controller-769875db56-nmdfz to ip-10-0-140-244.us-east-2.compute.internal 27m Normal Scheduled pod/model-serving-api-75bd89fdfc-gmb5t Successfully assigned opendatahub/model-serving-api-75bd89fdfc-gmb5t to ip-10-0-140-244.us-east-2.compute.internal 27m Normal Scheduled pod/kserve-controller-manager-776cc66d47-wssxm Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-wssxm to ip-10-0-132-179.us-east-2.compute.internal 28m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-j86rr Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-j86rr to ip-10-0-130-39.us-east-2.compute.internal 17m Normal Scheduled pod/maas-controller-dc84dbdd5-jss6g Successfully assigned opendatahub/maas-controller-dc84dbdd5-jss6g to ip-10-0-132-179.us-east-2.compute.internal 24m Normal Scheduled pod/ai-gateway-controller-5d9cd99f45-sv7sg Successfully assigned opendatahub/ai-gateway-controller-5d9cd99f45-sv7sg to ip-10-0-130-39.us-east-2.compute.internal 16m Normal Scheduled pod/maas-controller-dc84dbdd5-bt25k Successfully assigned opendatahub/maas-controller-dc84dbdd5-bt25k to ip-10-0-132-179.us-east-2.compute.internal 26m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-kwpvb Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-kwpvb to ip-10-0-132-179.us-east-2.compute.internal 27m Normal Scheduled pod/maas-controller-dc84dbdd5-9n7zf Successfully assigned opendatahub/maas-controller-dc84dbdd5-9n7zf to ip-10-0-132-179.us-east-2.compute.internal 27m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-6mw64 to ip-10-0-132-179.us-east-2.compute.internal 27m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 to ip-10-0-132-179.us-east-2.compute.internal 28m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 28m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-j86rr 28m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-j86rr Add eth0 [10.133.0.38/23] from ovn-kubernetes 28m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-j86rr Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 28m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-j86rr Container created 28m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-j86rr Container started 28m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-j86rr Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 3.594s (3.594s including waiting). Image size: 195729000 bytes. 27m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-j86rr_79f5e5a0-5fe0-4a3e-b68a-1e2c2503a4a2 became leader 27m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-579cf7c6d from 0 to 1 27m Normal SuccessfulCreate replicaset/maas-controller-579cf7c6d Created pod: maas-controller-579cf7c6d-ndnw7 27m Normal SuccessfulCreate replicaset/maas-controller-dc84dbdd5 Created pod: maas-controller-dc84dbdd5-9n7zf 27m Normal AddedInterface pod/maas-controller-dc84dbdd5-9n7zf Add eth0 [10.132.0.33/23] from ovn-kubernetes 27m Normal Pulling pod/maas-controller-579cf7c6d-ndnw7 Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 27m Normal AddedInterface pod/maas-controller-579cf7c6d-ndnw7 Add eth0 [10.132.0.32/23] from ovn-kubernetes 27m Normal Created pod/maas-controller-579cf7c6d-ndnw7 Container created 27m Normal Pulled pod/maas-controller-579cf7c6d-ndnw7 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.083s (3.083s including waiting). Image size: 200053271 bytes. 27m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-579cf7c6d-ndnw7_bf7518ec-daff-4b29-9a94-9dba4beb7231 became leader 27m Normal Pulled pod/maas-controller-dc84dbdd5-9n7zf Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.143s (3.143s including waiting). Image size: 200053271 bytes. 27m Normal Started pod/maas-controller-579cf7c6d-ndnw7 Container started 27m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 27m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 27m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 27m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 Add eth0 [10.132.0.34/23] from ovn-kubernetes 27m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 27m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 27m Normal Pulling pod/kserve-controller-manager-776cc66d47-wssxm Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 27m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-wssxm Add eth0 [10.132.0.35/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-wssxm 27m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 27m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 27m Normal Pulling pod/model-serving-api-75bd89fdfc-gmb5t Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 27m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 27m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 27m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-gmb5t 27m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-6mw64 27m Normal AddedInterface pod/model-serving-api-75bd89fdfc-gmb5t Add eth0 [10.134.0.31/23] from ovn-kubernetes 27m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Add eth0 [10.132.0.36/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-nmdfz 27m Normal AddedInterface pod/odh-model-controller-769875db56-nmdfz Add eth0 [10.134.0.32/23] from ovn-kubernetes 27m Normal Pulling pod/odh-model-controller-769875db56-nmdfz Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 27m Normal Started pod/model-serving-api-75bd89fdfc-gmb5t Container started 27m Normal Started pod/kserve-controller-manager-776cc66d47-wssxm Container started 27m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Container started 27m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Container created 27m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.726s (2.726s including waiting). Image size: 235100981 bytes. 27m Normal Pulled pod/kserve-controller-manager-776cc66d47-wssxm Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.947s (2.947s including waiting). Image size: 247676581 bytes. 27m Normal Created pod/model-serving-api-75bd89fdfc-gmb5t Container created 27m Normal Pulled pod/odh-model-controller-769875db56-nmdfz Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 2.9s (2.9s including waiting). Image size: 236755567 bytes. 27m Normal Created pod/kserve-controller-manager-776cc66d47-wssxm Container created 27m Normal Pulled pod/model-serving-api-75bd89fdfc-gmb5t Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.091s (2.091s including waiting). Image size: 184925761 bytes. 27m Normal Created pod/odh-model-controller-769875db56-nmdfz Container created 27m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-nmdfz_8319e54c-a275-4d0c-b386-92b0f5e033a4 became leader 27m Normal Started pod/odh-model-controller-769875db56-nmdfz Container started 27m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-wssxm_8c249326-9358-4eac-8802-ffd3f6b8864a became leader 27m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-579cf7c6d from 1 to 0 27m Normal Killing pod/maas-controller-579cf7c6d-ndnw7 Stopping container manager 27m Normal SuccessfulDelete replicaset/maas-controller-579cf7c6d Deleted pod: maas-controller-579cf7c6d-ndnw7 27m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-6mw64_62e2b365-df77-451c-a82b-2c01d3d4374e became leader 27m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.861s (13.861s including waiting). Image size: 1089502687 bytes. 27m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 Container created 27m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-w7lk7 Container started 26m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-dc84dbdd5-9n7zf_69f6c202-0f29-493e-a180-806532c8d8bc became leader 26m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 26m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-kwpvb 26m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-kwpvb Add eth0 [10.132.0.40/23] from ovn-kubernetes 26m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-kwpvb Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 26m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-kwpvb Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 390ms (390ms including waiting). Image size: 235100981 bytes. 26m Normal Started pod/llmisvc-controller-manager-6466c554d8-kwpvb Container started 26m Normal Created pod/llmisvc-controller-manager-6466c554d8-kwpvb Container created 25m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 25m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-6mw64 25m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-6mw64 Stopping container manager 25m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-kwpvb_033486fa-9ae2-41a1-8deb-5050cf19a1ae became leader 24m Normal Created pod/maas-controller-dc84dbdd5-9n7zf Container created 24m Normal Pulling pod/maas-controller-dc84dbdd5-9n7zf Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 24m Normal Pulled pod/maas-controller-dc84dbdd5-9n7zf Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 519ms (519ms including waiting). Image size: 200053271 bytes. 24m Normal Started pod/maas-controller-dc84dbdd5-9n7zf Container started 24m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-dc84dbdd5-9n7zf_4dedfbd3-ec77-4f35-8ce5-fa73fb013766 became leader 24m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-5d9cd99f45 from 0 to 1 24m Normal SuccessfulCreate replicaset/ai-gateway-controller-5d9cd99f45 Created pod: ai-gateway-controller-5d9cd99f45-sv7sg 24m Normal Pulling pod/ai-gateway-controller-5d9cd99f45-sv7sg Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" 24m Normal AddedInterface pod/ai-gateway-controller-5d9cd99f45-sv7sg Add eth0 [10.133.0.49/23] from ovn-kubernetes 24m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-5d9cd99f45-sv7sg_6018d5a3-63a9-4e03-93ac-2691ee31eebb became leader 24m Normal Pulled pod/ai-gateway-controller-5d9cd99f45-sv7sg Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" in 3.873s (3.873s including waiting). Image size: 184085929 bytes. 24m Normal Created pod/ai-gateway-controller-5d9cd99f45-sv7sg Container created 24m Normal Started pod/ai-gateway-controller-5d9cd99f45-sv7sg Container started 24m Normal AddedInterface pod/ai-gateway-controller-569c46f4cf-fzxcn Add eth0 [10.133.0.53/23] from ovn-kubernetes 24m Normal Pulling pod/ai-gateway-controller-569c46f4cf-fzxcn Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" 24m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-569c46f4cf from 0 to 1 24m Normal SuccessfulCreate replicaset/ai-gateway-controller-569c46f4cf Created pod: ai-gateway-controller-569c46f4cf-fzxcn 24m Normal Pulled pod/ai-gateway-controller-569c46f4cf-fzxcn Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-191" in 460ms (460ms including waiting). Image size: 184085929 bytes. 24m Normal Created pod/ai-gateway-controller-569c46f4cf-fzxcn Container created 24m Normal Started pod/ai-gateway-controller-569c46f4cf-fzxcn Container started 24m Normal Killing pod/ai-gateway-controller-5d9cd99f45-sv7sg Stopping container manager 24m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-5d9cd99f45 from 1 to 0 24m Normal SuccessfulDelete replicaset/ai-gateway-controller-5d9cd99f45 Deleted pod: ai-gateway-controller-5d9cd99f45-sv7sg 23m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-569c46f4cf-fzxcn_96b00a42-710b-428c-aa83-3411d9a66659 became leader 17m Normal Killing pod/maas-controller-dc84dbdd5-9n7zf Stopping container manager 17m Normal SuccessfulDelete replicaset/maas-controller-dc84dbdd5 Deleted pod: maas-controller-dc84dbdd5-9n7zf 17m Normal AddedInterface pod/maas-controller-dc84dbdd5-jss6g Add eth0 [10.132.0.80/23] from ovn-kubernetes 17m Normal SuccessfulCreate replicaset/maas-controller-dc84dbdd5 Created pod: maas-controller-dc84dbdd5-jss6g 17m Normal Pulling pod/maas-controller-dc84dbdd5-jss6g Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 17m Normal Started pod/maas-controller-dc84dbdd5-jss6g Container started 17m Normal Pulled pod/maas-controller-dc84dbdd5-jss6g Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 613ms (613ms including waiting). Image size: 200053271 bytes. 17m Normal Created pod/maas-controller-dc84dbdd5-jss6g Container created 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-dc84dbdd5-jss6g_905d7469-e3c8-4d19-95d4-795f63013901 became leader 16m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-dc84dbdd5 from 1 to 0 16m Normal SuccessfulDelete replicaset/maas-controller-dc84dbdd5 Deleted pod: maas-controller-dc84dbdd5-jss6g 16m Normal Killing pod/maas-controller-dc84dbdd5-jss6g Stopping container manager 16m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-dc84dbdd5 from 0 to 1 16m Normal SuccessfulCreate replicaset/maas-controller-dc84dbdd5 Created pod: maas-controller-dc84dbdd5-bt25k 16m Normal Pulling pod/maas-controller-dc84dbdd5-bt25k Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 16m Normal AddedInterface pod/maas-controller-dc84dbdd5-bt25k Add eth0 [10.132.0.81/23] from ovn-kubernetes 16m Normal Started pod/maas-controller-dc84dbdd5-bt25k Container started 16m Normal Created pod/maas-controller-dc84dbdd5-bt25k Container created 16m Normal Pulled pod/maas-controller-dc84dbdd5-bt25k Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 673ms (673ms including waiting). Image size: 200053271 bytes. 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-dc84dbdd5-bt25k_79f1629d-f1e7-4c41-be67-cc9491c8d0b3 became leader