=== events in opendatahub (2026-11-02T20:42:39Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 68m Normal Scheduled pod/model-serving-api-75bd89fdfc-phk29 Successfully assigned opendatahub/model-serving-api-75bd89fdfc-phk29 to ip-10-0-133-9.us-west-2.compute.internal 53m Normal Scheduled pod/test-curl-7137-30af7c Successfully assigned opendatahub/test-curl-7137-30af7c to ip-10-0-132-174.us-west-2.compute.internal 67m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-wcnxm Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-wcnxm to ip-10-0-132-174.us-west-2.compute.internal 64m Normal Scheduled pod/ai-gateway-controller-6c866c6d6d-md4dq Successfully assigned opendatahub/ai-gateway-controller-6c866c6d6d-md4dq to ip-10-0-131-244.us-west-2.compute.internal 68m Normal Scheduled pod/maas-controller-6ddc6f7dd5-t47p9 Successfully assigned opendatahub/maas-controller-6ddc6f7dd5-t47p9 to ip-10-0-132-174.us-west-2.compute.internal 68m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-qdfsz to ip-10-0-132-174.us-west-2.compute.internal 22m Normal Scheduled pod/maas-controller-6ddc6f7dd5-p4d2c Successfully assigned opendatahub/maas-controller-6ddc6f7dd5-p4d2c to ip-10-0-132-174.us-west-2.compute.internal 68m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb to ip-10-0-132-174.us-west-2.compute.internal 68m Normal Scheduled pod/maas-controller-579cf7c6d-h2bmf Successfully assigned opendatahub/maas-controller-579cf7c6d-h2bmf to ip-10-0-132-174.us-west-2.compute.internal 53m Normal Scheduled pod/test-curl-7137-b1d914 Successfully assigned opendatahub/test-curl-7137-b1d914 to ip-10-0-132-174.us-west-2.compute.internal 23m Normal Scheduled pod/maas-controller-6ddc6f7dd5-tdtgx Successfully assigned opendatahub/maas-controller-6ddc6f7dd5-tdtgx to ip-10-0-132-174.us-west-2.compute.internal 68m Normal Scheduled pod/odh-model-controller-769875db56-fnf5k Successfully assigned opendatahub/odh-model-controller-769875db56-fnf5k to ip-10-0-132-174.us-west-2.compute.internal 68m Normal Scheduled pod/kserve-controller-manager-776cc66d47-pxv8z Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-pxv8z to ip-10-0-132-174.us-west-2.compute.internal 65m Normal Scheduled pod/ai-gateway-controller-55cd769764-fjcr4 Successfully assigned opendatahub/ai-gateway-controller-55cd769764-fjcr4 to ip-10-0-131-244.us-west-2.compute.internal 69m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-4sl5m Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-4sl5m to ip-10-0-132-174.us-west-2.compute.internal 69m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-4sl5m Add eth0 [10.132.0.46/23] from ovn-kubernetes 69m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 69m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-4sl5m Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 69m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-4sl5m 69m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-4sl5m Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.898s (2.898s including waiting). Image size: 195729000 bytes. 69m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-4sl5m Container created 69m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-4sl5m Container started 69m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-4sl5m_11c8ad84-8a4a-49de-a82f-8a808ffef0fd became leader 68m Normal SuccessfulCreate replicaset/maas-controller-579cf7c6d Created pod: maas-controller-579cf7c6d-h2bmf 68m Normal SuccessfulCreate replicaset/maas-controller-6ddc6f7dd5 Created pod: maas-controller-6ddc6f7dd5-t47p9 68m Warning FailedMount pod/maas-controller-579cf7c6d-h2bmf MountVolume.SetUp failed for volume "webhook-cert" : secret "maas-controller-webhook-cert" not found 68m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-579cf7c6d from 0 to 1 68m Normal AddedInterface pod/maas-controller-6ddc6f7dd5-t47p9 Add eth0 [10.132.0.48/23] from ovn-kubernetes 68m Normal Pulling pod/maas-controller-6ddc6f7dd5-t47p9 Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 68m Normal AddedInterface pod/maas-controller-579cf7c6d-h2bmf Add eth0 [10.132.0.47/23] from ovn-kubernetes 68m Normal Pulling pod/maas-controller-579cf7c6d-h2bmf Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 68m Normal Pulled pod/maas-controller-579cf7c6d-h2bmf Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.383s (3.383s including waiting). Image size: 200053271 bytes. 68m Normal Created pod/maas-controller-6ddc6f7dd5-t47p9 Container created 68m Normal Pulled pod/maas-controller-6ddc6f7dd5-t47p9 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.692s (3.692s including waiting). Image size: 200053271 bytes. 68m Normal Created pod/maas-controller-579cf7c6d-h2bmf Container created 68m Normal Started pod/maas-controller-579cf7c6d-h2bmf Container started 68m Normal Started pod/maas-controller-6ddc6f7dd5-t47p9 Container started 68m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-6ddc6f7dd5-t47p9_150e1a88-0f68-40c4-a113-a0606764b835 became leader 68m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-rvzdb 68m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 68m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found 68m Warning FailedMount pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb MountVolume.SetUp failed for volume "tls-cert" : secret "usage-logs-tenancy-proxy-tls" not found 68m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb Add eth0 [10.132.0.49/23] from ovn-kubernetes 68m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 68m Normal SuccessfulDelete replicaset/maas-controller-579cf7c6d Deleted pod: maas-controller-579cf7c6d-h2bmf 68m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-579cf7c6d from 1 to 0 68m Normal Killing pod/maas-controller-579cf7c6d-h2bmf Stopping container manager 68m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb Container started 68m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb Container created 68m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-rvzdb Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.199s (13.199s including waiting). Image size: 1089502687 bytes. 68m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 68m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 68m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-fnf5k 68m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-pxv8z Add eth0 [10.132.0.50/23] from ovn-kubernetes 68m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-pxv8z 68m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-qdfsz 68m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 68m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 68m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-phk29 68m Normal AddedInterface pod/model-serving-api-75bd89fdfc-phk29 Add eth0 [10.133.0.23/23] from ovn-kubernetes 68m Normal Pulling pod/model-serving-api-75bd89fdfc-phk29 Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 68m Normal Pulling pod/kserve-controller-manager-776cc66d47-pxv8z Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 68m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 68m Normal AddedInterface pod/odh-model-controller-769875db56-fnf5k Add eth0 [10.132.0.52/23] from ovn-kubernetes 68m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Add eth0 [10.132.0.51/23] from ovn-kubernetes 68m Normal Pulling pod/odh-model-controller-769875db56-fnf5k Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 68m Normal Created pod/model-serving-api-75bd89fdfc-phk29 Container created 68m Normal Pulled pod/model-serving-api-75bd89fdfc-phk29 Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.61s (2.61s including waiting). Image size: 184925761 bytes. 68m Normal Started pod/model-serving-api-75bd89fdfc-phk29 Container started 68m Normal Created pod/kserve-controller-manager-776cc66d47-pxv8z Container created 68m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.729s (3.729s including waiting). Image size: 235100981 bytes. 68m Normal Pulled pod/kserve-controller-manager-776cc66d47-pxv8z Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.566s (3.566s including waiting). Image size: 247676581 bytes. 68m Normal Started pod/kserve-controller-manager-776cc66d47-pxv8z Container started 68m Normal Pulled pod/odh-model-controller-769875db56-fnf5k Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.243s (4.243s including waiting). Image size: 236755567 bytes. 68m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Container started 68m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-pxv8z_67f44362-9b66-4b78-9dfe-7f98f3690604 became leader 68m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Container created 68m Normal Started pod/odh-model-controller-769875db56-fnf5k Container started 68m Normal Created pod/odh-model-controller-769875db56-fnf5k Container created 68m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-qdfsz_529008ca-91d1-4bb8-beec-94bf3f778c70 became leader 68m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-fnf5k_b288e067-11c5-4f16-b170-f49969905741 became leader 67m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 67m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-wcnxm 67m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-wcnxm Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 67m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-wcnxm Add eth0 [10.132.0.56/23] from ovn-kubernetes 67m Normal Created pod/llmisvc-controller-manager-6466c554d8-wcnxm Container created 67m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-wcnxm Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 786ms (786ms including waiting). Image size: 235100981 bytes. 67m Normal Started pod/llmisvc-controller-manager-6466c554d8-wcnxm Container started 66m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 66m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-qdfsz 66m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-qdfsz Stopping container manager 66m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-wcnxm_a4b82585-971c-4b2b-a382-ff6caa03054f became leader 65m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-55cd769764 from 0 to 1 65m Normal SuccessfulCreate replicaset/ai-gateway-controller-55cd769764 Created pod: ai-gateway-controller-55cd769764-fjcr4 65m Normal AddedInterface pod/ai-gateway-controller-55cd769764-fjcr4 Add eth0 [10.134.0.37/23] from ovn-kubernetes 65m Normal Pulling pod/ai-gateway-controller-55cd769764-fjcr4 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" 65m Normal Pulled pod/ai-gateway-controller-55cd769764-fjcr4 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 3.665s (3.665s including waiting). Image size: 183991737 bytes. 65m Normal Created pod/ai-gateway-controller-55cd769764-fjcr4 Container created 65m Normal Started pod/ai-gateway-controller-55cd769764-fjcr4 Container started 65m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-55cd769764-fjcr4_9971a365-5768-4f78-88bf-ebda00d8d41d became leader 64m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-6c866c6d6d from 0 to 1 64m Normal SuccessfulCreate replicaset/ai-gateway-controller-6c866c6d6d Created pod: ai-gateway-controller-6c866c6d6d-md4dq 64m Normal AddedInterface pod/ai-gateway-controller-6c866c6d6d-md4dq Add eth0 [10.134.0.39/23] from ovn-kubernetes 64m Normal Pulling pod/ai-gateway-controller-6c866c6d6d-md4dq Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" 64m Normal Pulled pod/ai-gateway-controller-6c866c6d6d-md4dq Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 828ms (828ms including waiting). Image size: 183991737 bytes. 64m Normal Created pod/ai-gateway-controller-6c866c6d6d-md4dq Container created 64m Normal Started pod/ai-gateway-controller-6c866c6d6d-md4dq Container started 64m Normal Killing pod/ai-gateway-controller-55cd769764-fjcr4 Stopping container manager 64m Normal SuccessfulDelete replicaset/ai-gateway-controller-55cd769764 Deleted pod: ai-gateway-controller-55cd769764-fjcr4 64m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-55cd769764 from 1 to 0 64m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-6c866c6d6d-md4dq_af76bdb9-f97d-4ebc-af8c-c4b44843abad became leader 53m Normal Pulling pod/test-curl-7137-30af7c Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 53m Normal AddedInterface pod/test-curl-7137-30af7c Add eth0 [10.132.0.144/23] from ovn-kubernetes 53m Normal Started pod/test-curl-7137-30af7c Container started 53m Normal Pulled pod/test-curl-7137-30af7c Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 1.295s (1.295s including waiting). Image size: 106100986 bytes. 53m Normal Created pod/test-curl-7137-30af7c Container created 53m Normal Killing pod/test-curl-7137-30af7c Stopping container test-curl-7137-30af7c 53m Normal Started pod/test-curl-7137-b1d914 Container started 53m Normal Created pod/test-curl-7137-b1d914 Container created 53m Normal Pulled pod/test-curl-7137-b1d914 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/test-curl-7137-b1d914 Add eth0 [10.132.0.147/23] from ovn-kubernetes 53m Normal Killing pod/test-curl-7137-b1d914 Stopping container test-curl-7137-b1d914 23m Normal SuccessfulDelete replicaset/maas-controller-6ddc6f7dd5 Deleted pod: maas-controller-6ddc6f7dd5-t47p9 23m Normal Killing pod/maas-controller-6ddc6f7dd5-t47p9 Stopping container manager 23m Normal SuccessfulCreate replicaset/maas-controller-6ddc6f7dd5 Created pod: maas-controller-6ddc6f7dd5-tdtgx 23m Normal AddedInterface pod/maas-controller-6ddc6f7dd5-tdtgx Add eth0 [10.132.1.16/23] from ovn-kubernetes 23m Normal Pulling pod/maas-controller-6ddc6f7dd5-tdtgx Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 23m Normal Created pod/maas-controller-6ddc6f7dd5-tdtgx Container created 23m Normal Started pod/maas-controller-6ddc6f7dd5-tdtgx Container started 23m Normal Pulled pod/maas-controller-6ddc6f7dd5-tdtgx Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 1.006s (1.006s including waiting). Image size: 200053271 bytes. 23m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-6ddc6f7dd5-tdtgx_8dd7c1a2-74b1-4a2f-a4b6-1c483428017c became leader 22m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-6ddc6f7dd5 from 1 to 0 22m Normal SuccessfulDelete replicaset/maas-controller-6ddc6f7dd5 Deleted pod: maas-controller-6ddc6f7dd5-tdtgx 22m Normal Killing pod/maas-controller-6ddc6f7dd5-tdtgx Stopping container manager 22m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-6ddc6f7dd5 from 0 to 1 22m Normal SuccessfulCreate replicaset/maas-controller-6ddc6f7dd5 Created pod: maas-controller-6ddc6f7dd5-p4d2c 22m Normal Pulling pod/maas-controller-6ddc6f7dd5-p4d2c Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 22m Normal AddedInterface pod/maas-controller-6ddc6f7dd5-p4d2c Add eth0 [10.132.1.17/23] from ovn-kubernetes 22m Normal Started pod/maas-controller-6ddc6f7dd5-p4d2c Container started 22m Normal Created pod/maas-controller-6ddc6f7dd5-p4d2c Container created 22m Normal Pulled pod/maas-controller-6ddc6f7dd5-p4d2c Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 860ms (860ms including waiting). Image size: 200053271 bytes. 22m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-6ddc6f7dd5-p4d2c_56a4ffa6-dd2e-442f-ba2c-cbb0e3a4bb63 became leader