=== events in opendatahub (2026-11-04T05:10:42Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 13m Normal Scheduled pod/ai-gateway-controller-64466f8c8d-bg6dc Successfully assigned opendatahub/ai-gateway-controller-64466f8c8d-bg6dc to ip-10-0-133-180.us-west-1.compute.internal 17m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d to ip-10-0-135-45.us-west-1.compute.internal 6m4s Normal Scheduled pod/test-curl-8034-83b602 Successfully assigned opendatahub/test-curl-8034-83b602 to ip-10-0-135-45.us-west-1.compute.internal 7m13s Normal Scheduled pod/test-curl-8034-7d9864 Successfully assigned opendatahub/test-curl-8034-7d9864 to ip-10-0-135-45.us-west-1.compute.internal 6m7s Normal Scheduled pod/test-curl-8034-74c00b Successfully assigned opendatahub/test-curl-8034-74c00b to ip-10-0-135-45.us-west-1.compute.internal 16m Normal Scheduled pod/odh-model-controller-769875db56-qprw2 Successfully assigned opendatahub/odh-model-controller-769875db56-qprw2 to ip-10-0-141-132.us-west-1.compute.internal 16m Normal Scheduled pod/model-serving-api-75bd89fdfc-7pj6b Successfully assigned opendatahub/model-serving-api-75bd89fdfc-7pj6b to ip-10-0-133-180.us-west-1.compute.internal 13m Normal Scheduled pod/ai-gateway-controller-74f5787476-n6s84 Successfully assigned opendatahub/ai-gateway-controller-74f5787476-n6s84 to ip-10-0-141-132.us-west-1.compute.internal 16m Normal Scheduled pod/kserve-controller-manager-776cc66d47-rdlg7 Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-rdlg7 to ip-10-0-135-45.us-west-1.compute.internal 16m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-lq652 Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-lq652 to ip-10-0-135-45.us-west-1.compute.internal 17m Normal Scheduled pod/maas-controller-8b95c9c66-96rnk Successfully assigned opendatahub/maas-controller-8b95c9c66-96rnk to ip-10-0-135-45.us-west-1.compute.internal 17m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-glcvz Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-glcvz to ip-10-0-133-180.us-west-1.compute.internal 17m Normal Scheduled pod/maas-controller-c56fdf97-qvkx4 Successfully assigned opendatahub/maas-controller-c56fdf97-qvkx4 to ip-10-0-135-45.us-west-1.compute.internal 15m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-vh4c8 Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-vh4c8 to ip-10-0-135-45.us-west-1.compute.internal 17m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 17m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-glcvz 17m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-glcvz Add eth0 [10.133.0.25/23] from ovn-kubernetes 17m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-glcvz Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 17m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-glcvz Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.764s (1.764s including waiting). Image size: 195729000 bytes. 17m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-glcvz Container created 17m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-glcvz Container started 17m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-glcvz_b572d7fc-e5c8-4af5-b561-ed80bffcd1ef became leader 17m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-8b95c9c66 from 0 to 1 17m Normal AddedInterface pod/maas-controller-8b95c9c66-96rnk Add eth0 [10.132.0.49/23] from ovn-kubernetes 17m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-c56fdf97 from 0 to 1 17m Normal SuccessfulCreate replicaset/maas-controller-8b95c9c66 Created pod: maas-controller-8b95c9c66-96rnk 17m Normal SuccessfulCreate replicaset/maas-controller-c56fdf97 Created pod: maas-controller-c56fdf97-qvkx4 17m Normal Pulling pod/maas-controller-8b95c9c66-96rnk Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 17m Normal AddedInterface pod/maas-controller-c56fdf97-qvkx4 Add eth0 [10.132.0.50/23] from ovn-kubernetes 17m Normal Created pod/maas-controller-8b95c9c66-96rnk Container created 17m Normal Started pod/maas-controller-8b95c9c66-96rnk Container started 17m Normal Pulled pod/maas-controller-8b95c9c66-96rnk Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 3.459s (3.459s including waiting). Image size: 200103442 bytes. 17m Normal Pulled pod/maas-controller-c56fdf97-qvkx4 Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 3.144s (3.144s including waiting). Image size: 200103442 bytes. 17m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-8b95c9c66-96rnk_fdb4a82d-e861-43d7-b2bf-7db08948300a became leader 17m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 17m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 17m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 17m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-zvm6d 17m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d Add eth0 [10.132.0.51/23] from ovn-kubernetes 17m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 16m Normal SuccessfulDelete replicaset/maas-controller-8b95c9c66 Deleted pod: maas-controller-8b95c9c66-96rnk 16m Normal Killing pod/maas-controller-8b95c9c66-96rnk Stopping container manager 16m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-8b95c9c66 from 1 to 0 16m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d Container started 16m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 12.419s (12.419s including waiting). Image size: 1089502687 bytes. 16m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-zvm6d Container created 16m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 16m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-rdlg7 16m Normal Pulling pod/kserve-controller-manager-776cc66d47-rdlg7 Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 16m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 16m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-lq652 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 16m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-qprw2 16m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 16m Normal Pulling pod/model-serving-api-75bd89fdfc-7pj6b Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 16m Normal AddedInterface pod/odh-model-controller-769875db56-qprw2 Add eth0 [10.134.0.30/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-lq652 16m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-rdlg7 Add eth0 [10.132.0.52/23] from ovn-kubernetes 16m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 16m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-lq652 Add eth0 [10.132.0.53/23] from ovn-kubernetes 16m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-7pj6b 16m Normal AddedInterface pod/model-serving-api-75bd89fdfc-7pj6b Add eth0 [10.133.0.27/23] from ovn-kubernetes 16m Normal Pulling pod/odh-model-controller-769875db56-qprw2 Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 16m Normal Created pod/model-serving-api-75bd89fdfc-7pj6b Container created 16m Normal Pulled pod/model-serving-api-75bd89fdfc-7pj6b Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.134s (2.134s including waiting). Image size: 184925761 bytes. 16m Normal Started pod/model-serving-api-75bd89fdfc-7pj6b Container started 16m Normal Created pod/odh-model-controller-769875db56-qprw2 Container created 16m Normal Pulled pod/odh-model-controller-769875db56-qprw2 Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 2.894s (2.894s including waiting). Image size: 236755567 bytes. 16m Normal Started pod/odh-model-controller-769875db56-qprw2 Container started 16m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-lq652 Container started 16m Normal Pulled pod/kserve-controller-manager-776cc66d47-rdlg7 Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.373s (3.373s including waiting). Image size: 247676581 bytes. 16m Normal Created pod/kserve-controller-manager-776cc66d47-rdlg7 Container created 16m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-qprw2_27a08e18-dcaf-4513-8662-963c2cd05d49 became leader 16m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-lq652 Container created 16m Normal Started pod/kserve-controller-manager-776cc66d47-rdlg7 Container started 16m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-lq652 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.36s (3.36s including waiting). Image size: 235100981 bytes. 16m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-rdlg7_0de535c2-20b8-42ad-869e-9650d773dfae became leader 16m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-lq652_ee83d506-1f27-4c14-88ff-f2f612bd6af5 became leader 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-c56fdf97-qvkx4_d2f638a6-d311-43a2-9744-4712f8428803 became leader 15m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 15m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-vh4c8 15m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-vh4c8 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 15m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-vh4c8 Add eth0 [10.132.0.55/23] from ovn-kubernetes 15m Normal Created pod/llmisvc-controller-manager-6466c554d8-vh4c8 Container created 15m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-vh4c8 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 739ms (739ms including waiting). Image size: 235100981 bytes. 15m Normal Started pod/llmisvc-controller-manager-6466c554d8-vh4c8 Container started 15m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 15m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-lq652 15m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-lq652 Stopping container manager 14m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-vh4c8_e83a45f6-ccf4-4b41-82b0-2d646fc8f3f6 became leader 14m Normal Pulling pod/maas-controller-c56fdf97-qvkx4 Pulling image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" 14m Normal Pulled pod/maas-controller-c56fdf97-qvkx4 Successfully pulled image "quay.io/opendatahub/maas-controller:08858542cd3bba18175e02ba9b8418547f22ade2" in 889ms (889ms including waiting). Image size: 200103442 bytes. 14m Normal Created pod/maas-controller-c56fdf97-qvkx4 Container created 14m Normal Started pod/maas-controller-c56fdf97-qvkx4 Container started 14m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-c56fdf97-qvkx4_1e9abbc0-9a51-47cb-9620-6b22c6b30731 became leader 13m Normal Pulling pod/ai-gateway-controller-74f5787476-n6s84 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-215" 13m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-74f5787476 from 0 to 1 13m Normal AddedInterface pod/ai-gateway-controller-74f5787476-n6s84 Add eth0 [10.134.0.38/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/ai-gateway-controller-74f5787476 Created pod: ai-gateway-controller-74f5787476-n6s84 13m Normal Started pod/ai-gateway-controller-74f5787476-n6s84 Container started 13m Normal Pulled pod/ai-gateway-controller-74f5787476-n6s84 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-215" in 6.438s (6.438s including waiting). Image size: 184156602 bytes. 13m Normal Created pod/ai-gateway-controller-74f5787476-n6s84 Container created 13m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-74f5787476-n6s84_3eb06b8e-db44-4d7b-b9ef-c9d5f2b35f7b became leader 13m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-64466f8c8d from 0 to 1 13m Normal SuccessfulCreate replicaset/ai-gateway-controller-64466f8c8d Created pod: ai-gateway-controller-64466f8c8d-bg6dc 13m Normal Pulling pod/ai-gateway-controller-64466f8c8d-bg6dc Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-215" 13m Normal AddedInterface pod/ai-gateway-controller-64466f8c8d-bg6dc Add eth0 [10.133.0.38/23] from ovn-kubernetes 13m Normal Started pod/ai-gateway-controller-64466f8c8d-bg6dc Container started 13m Normal Created pod/ai-gateway-controller-64466f8c8d-bg6dc Container created 13m Normal Pulled pod/ai-gateway-controller-64466f8c8d-bg6dc Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-215" in 2.699s (2.699s including waiting). Image size: 184156602 bytes. 13m Normal SuccessfulDelete replicaset/ai-gateway-controller-74f5787476 Deleted pod: ai-gateway-controller-74f5787476-n6s84 13m Normal Killing pod/ai-gateway-controller-74f5787476-n6s84 Stopping container manager 13m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-74f5787476 from 1 to 0 13m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-64466f8c8d-bg6dc_8bf34423-99f6-4fd0-9903-dd31177f6ec1 became leader 7m13s Normal AddedInterface pod/test-curl-8034-7d9864 Add eth0 [10.132.0.72/23] from ovn-kubernetes 7m13s Normal Pulling pod/test-curl-8034-7d9864 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 7m11s Normal Started pod/test-curl-8034-7d9864 Container started 7m11s Normal Pulled pod/test-curl-8034-7d9864 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.472s (2.472s including waiting). Image size: 106100986 bytes. 7m11s Normal Created pod/test-curl-8034-7d9864 Container created 6m9s Normal Killing pod/test-curl-8034-7d9864 Stopping container test-curl-8034-7d9864 6m7s Normal Started pod/test-curl-8034-74c00b Container started 6m7s Normal Created pod/test-curl-8034-74c00b Container created 6m7s Normal Pulled pod/test-curl-8034-74c00b Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 6m7s Normal AddedInterface pod/test-curl-8034-74c00b Add eth0 [10.132.0.74/23] from ovn-kubernetes 6m4s Normal AddedInterface pod/test-curl-8034-83b602 Add eth0 [10.132.0.75/23] from ovn-kubernetes 6m4s Normal Pulled pod/test-curl-8034-83b602 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 6m4s Normal Started pod/test-curl-8034-83b602 Container started 6m4s Normal Killing pod/test-curl-8034-74c00b Stopping container test-curl-8034-74c00b 6m4s Normal Created pod/test-curl-8034-83b602 Container created 6m1s Normal Killing pod/test-curl-8034-83b602 Stopping container test-curl-8034-83b602