=== events in opendatahub (2026-11-03T08:46:11Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 35m Normal Scheduled pod/test-curl-6985-3fd5e5 Successfully assigned opendatahub/test-curl-6985-3fd5e5 to ip-10-0-140-232.us-east-2.compute.internal 29m Normal Scheduled pod/odh-model-controller-769875db56-fzvxh Successfully assigned opendatahub/odh-model-controller-769875db56-fzvxh to ip-10-0-143-47.us-east-2.compute.internal 44m Normal Scheduled pod/ai-gateway-controller-6c56f47d8c-lv8cv Successfully assigned opendatahub/ai-gateway-controller-6c56f47d8c-lv8cv to ip-10-0-143-47.us-east-2.compute.internal 46m Normal Scheduled pod/model-serving-api-75bd89fdfc-mdwfw Successfully assigned opendatahub/model-serving-api-75bd89fdfc-mdwfw to ip-10-0-140-123.us-east-2.compute.internal 46m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-tgbhc to ip-10-0-140-232.us-east-2.compute.internal 16m Normal Scheduled pod/maas-controller-799454fcf4-2rncm Successfully assigned opendatahub/maas-controller-799454fcf4-2rncm to ip-10-0-140-232.us-east-2.compute.internal 45m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-r57t2 Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-r57t2 to ip-10-0-140-232.us-east-2.compute.internal 44m Normal Scheduled pod/ai-gateway-controller-7cb8f6bb64-85z6q Successfully assigned opendatahub/ai-gateway-controller-7cb8f6bb64-85z6q to ip-10-0-143-47.us-east-2.compute.internal 46m Normal Scheduled pod/maas-controller-799454fcf4-24ntl Successfully assigned opendatahub/maas-controller-799454fcf4-24ntl to ip-10-0-143-47.us-east-2.compute.internal 46m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 to ip-10-0-143-47.us-east-2.compute.internal 38m Normal Scheduled pod/test-curl-6994-fb05bd Successfully assigned opendatahub/test-curl-6994-fb05bd to ip-10-0-140-232.us-east-2.compute.internal 38m Normal Scheduled pod/test-curl-6994-4504b5 Successfully assigned opendatahub/test-curl-6994-4504b5 to ip-10-0-140-232.us-east-2.compute.internal 35m Normal Scheduled pod/test-curl-6985-eb56da Successfully assigned opendatahub/test-curl-6985-eb56da to ip-10-0-140-232.us-east-2.compute.internal 35m Normal Scheduled pod/test-curl-6985-e9d00c Successfully assigned opendatahub/test-curl-6985-e9d00c to ip-10-0-140-232.us-east-2.compute.internal 17m Normal Scheduled pod/maas-controller-799454fcf4-t4cpk Successfully assigned opendatahub/maas-controller-799454fcf4-t4cpk to ip-10-0-143-47.us-east-2.compute.internal 32m Normal Scheduled pod/test-curl-6982-fb3e29 Successfully assigned opendatahub/test-curl-6982-fb3e29 to ip-10-0-140-232.us-east-2.compute.internal 47m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-p6rtl Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-p6rtl to ip-10-0-140-232.us-east-2.compute.internal 34m Normal Scheduled pod/test-curl-6982-8bd389 Successfully assigned opendatahub/test-curl-6982-8bd389 to ip-10-0-140-232.us-east-2.compute.internal 46m Normal Scheduled pod/maas-controller-579cf7c6d-ctmbl Successfully assigned opendatahub/maas-controller-579cf7c6d-ctmbl to ip-10-0-143-47.us-east-2.compute.internal 35m Normal Scheduled pod/test-curl-6985-2630f8 Successfully assigned opendatahub/test-curl-6985-2630f8 to ip-10-0-140-232.us-east-2.compute.internal 46m Normal Scheduled pod/odh-model-controller-769875db56-8kjgc Successfully assigned opendatahub/odh-model-controller-769875db56-8kjgc to ip-10-0-143-47.us-east-2.compute.internal 46m Normal Scheduled pod/kserve-controller-manager-776cc66d47-v6gdq Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-v6gdq to ip-10-0-140-232.us-east-2.compute.internal 47m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 47m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-p6rtl 47m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-p6rtl Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 47m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-p6rtl Add eth0 [10.133.0.19/23] from ovn-kubernetes 47m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-p6rtl Container started 47m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-p6rtl Container created 47m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-p6rtl Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.03s (2.03s including waiting). Image size: 195729000 bytes. 47m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-p6rtl_31a257ae-6e0f-4569-90d2-c18b0a092757 became leader 46m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-mdwfw 46m Warning FailedMount pod/odh-model-controller-769875db56-8kjgc MountVolume.SetUp failed for volume "metrics-certs" : secret "odh-model-controller-metrics-tls" not found 46m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-v6gdq 46m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-tgbhc 46m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 46m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 46m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 46m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 46m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-8kjgc 46m Normal Pulling pod/kserve-controller-manager-776cc66d47-v6gdq Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 46m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Add eth0 [10.133.0.25/23] from ovn-kubernetes 46m Normal Pulling pod/model-serving-api-75bd89fdfc-mdwfw Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 46m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 46m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-v6gdq Add eth0 [10.133.0.24/23] from ovn-kubernetes 46m Normal AddedInterface pod/model-serving-api-75bd89fdfc-mdwfw Add eth0 [10.132.0.47/23] from ovn-kubernetes 46m Normal Pulling pod/odh-model-controller-769875db56-8kjgc Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 46m Normal AddedInterface pod/odh-model-controller-769875db56-8kjgc Add eth0 [10.134.0.33/23] from ovn-kubernetes 46m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-v6gdq_01e9a49a-3eec-459f-9fcf-808d522299b3 became leader 46m Normal Started pod/kserve-controller-manager-776cc66d47-v6gdq Container started 46m Normal Created pod/kserve-controller-manager-776cc66d47-v6gdq Container created 46m Normal Pulled pod/kserve-controller-manager-776cc66d47-v6gdq Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.905s (2.905s including waiting). Image size: 247676581 bytes. 46m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.385s (3.385s including waiting). Image size: 235100981 bytes. 46m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Container created 46m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Container started 46m Normal Started pod/odh-model-controller-769875db56-8kjgc Container started 46m Normal Created pod/odh-model-controller-769875db56-8kjgc Container created 46m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-tgbhc_5663d2fd-e331-42ca-a025-3af611972e93 became leader 46m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-8kjgc_202634e8-a22b-4e74-91c1-164d48550447 became leader 46m Normal Pulled pod/odh-model-controller-769875db56-8kjgc Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 4.388s (4.388s including waiting). Image size: 236755567 bytes. 46m Normal Pulled pod/model-serving-api-75bd89fdfc-mdwfw Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 7.225s (7.225s including waiting). Image size: 184925761 bytes. 46m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-579cf7c6d from 0 to 1 46m Normal SuccessfulCreate replicaset/maas-controller-799454fcf4 Created pod: maas-controller-799454fcf4-24ntl 46m Normal SuccessfulCreate replicaset/maas-controller-579cf7c6d Created pod: maas-controller-579cf7c6d-ctmbl 46m Normal AddedInterface pod/maas-controller-799454fcf4-24ntl Add eth0 [10.134.0.35/23] from ovn-kubernetes 46m Normal Pulling pod/maas-controller-579cf7c6d-ctmbl Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 46m Normal AddedInterface pod/maas-controller-579cf7c6d-ctmbl Add eth0 [10.134.0.34/23] from ovn-kubernetes 46m Normal Pulling pod/maas-controller-799454fcf4-24ntl Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 46m Normal Created pod/maas-controller-579cf7c6d-ctmbl Container created 46m Normal Started pod/maas-controller-579cf7c6d-ctmbl Container started 46m Normal Pulled pod/maas-controller-579cf7c6d-ctmbl Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.614s (3.614s including waiting). Image size: 200053271 bytes. 46m Normal Pulled pod/maas-controller-799454fcf4-24ntl Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 3.677s (3.677s including waiting). Image size: 200053271 bytes. 46m Normal Created pod/maas-controller-799454fcf4-24ntl Container created 46m Normal Started pod/maas-controller-799454fcf4-24ntl Container started 46m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-579cf7c6d-ctmbl_0d959751-8bd3-4853-b21e-9725dea0c831 became leader 46m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 46m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-rxd77 46m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 Add eth0 [10.134.0.36/23] from ovn-kubernetes 46m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 46m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-579cf7c6d from 1 to 0 46m Normal SuccessfulDelete replicaset/maas-controller-579cf7c6d Deleted pod: maas-controller-579cf7c6d-ctmbl 46m Normal Killing pod/maas-controller-579cf7c6d-ctmbl Stopping container manager 46m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 Container created 46m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 Container started 46m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-rxd77 Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 12.206s (12.206s including waiting). Image size: 1089502687 bytes. 45m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-799454fcf4-24ntl_04f63e9d-ffa5-466d-bcb9-6991e6f9d121 became leader 45m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-r57t2 Add eth0 [10.133.0.28/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 45m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-r57t2 45m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-r57t2 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 45m Normal Created pod/llmisvc-controller-manager-6466c554d8-r57t2 Container created 45m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-r57t2 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 569ms (569ms including waiting). Image size: 235100981 bytes. 45m Normal Started pod/llmisvc-controller-manager-6466c554d8-r57t2 Container started 45m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-tgbhc Stopping container manager 45m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 45m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-tgbhc 44m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-r57t2_0064b444-8342-4d69-b428-e794facd994e became leader 44m Normal SuccessfulCreate replicaset/ai-gateway-controller-7cb8f6bb64 Created pod: ai-gateway-controller-7cb8f6bb64-85z6q 44m Normal AddedInterface pod/ai-gateway-controller-7cb8f6bb64-85z6q Add eth0 [10.134.0.41/23] from ovn-kubernetes 44m Normal Pulling pod/ai-gateway-controller-7cb8f6bb64-85z6q Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-stable" 44m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-7cb8f6bb64 from 0 to 1 44m Normal Pulled pod/ai-gateway-controller-7cb8f6bb64-85z6q Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-stable" in 2.588s (2.588s including waiting). Image size: 183979434 bytes. 44m Normal Started pod/ai-gateway-controller-7cb8f6bb64-85z6q Container started 44m Normal Created pod/ai-gateway-controller-7cb8f6bb64-85z6q Container created 44m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-7cb8f6bb64-85z6q_09b43a57-491e-4ad1-8971-fb752ffcd52d became leader 44m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-6c56f47d8c from 0 to 1 44m Normal SuccessfulCreate replicaset/ai-gateway-controller-6c56f47d8c Created pod: ai-gateway-controller-6c56f47d8c-lv8cv 44m Normal Started pod/ai-gateway-controller-6c56f47d8c-lv8cv Container started 44m Normal Pulling pod/ai-gateway-controller-6c56f47d8c-lv8cv Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-stable" 44m Normal AddedInterface pod/ai-gateway-controller-6c56f47d8c-lv8cv Add eth0 [10.134.0.45/23] from ovn-kubernetes 44m Normal Pulled pod/ai-gateway-controller-6c56f47d8c-lv8cv Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-stable" in 479ms (479ms including waiting). Image size: 183979434 bytes. 44m Normal Created pod/ai-gateway-controller-6c56f47d8c-lv8cv Container created 43m Normal Killing pod/ai-gateway-controller-7cb8f6bb64-85z6q Stopping container manager 43m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-7cb8f6bb64 from 1 to 0 43m Normal SuccessfulDelete replicaset/ai-gateway-controller-7cb8f6bb64 Deleted pod: ai-gateway-controller-7cb8f6bb64-85z6q 43m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-6c56f47d8c-lv8cv_d8030c98-9f3d-43f7-9efa-705a46ee1593 became leader 38m Normal AddedInterface pod/test-curl-6994-fb05bd Add eth0 [10.133.0.68/23] from ovn-kubernetes 38m Normal Pulling pod/test-curl-6994-fb05bd Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 38m Normal Started pod/test-curl-6994-fb05bd Container started 38m Normal Created pod/test-curl-6994-fb05bd Container created 38m Normal Pulled pod/test-curl-6994-fb05bd Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.334s (3.334s including waiting). Image size: 106100986 bytes. 38m Normal Pulled pod/test-curl-6994-4504b5 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 38m Normal Started pod/test-curl-6994-4504b5 Container started 38m Normal Created pod/test-curl-6994-4504b5 Container created 38m Normal AddedInterface pod/test-curl-6994-4504b5 Add eth0 [10.133.0.70/23] from ovn-kubernetes 38m Normal Killing pod/test-curl-6994-fb05bd Stopping container test-curl-6994-fb05bd 38m Normal Killing pod/test-curl-6994-4504b5 Stopping container test-curl-6994-4504b5 35m Normal Created pod/test-curl-6985-e9d00c Container created 35m Normal Pulled pod/test-curl-6985-e9d00c Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/test-curl-6985-e9d00c Add eth0 [10.133.0.99/23] from ovn-kubernetes 35m Normal Started pod/test-curl-6985-e9d00c Container started 35m Normal Started pod/test-curl-6985-eb56da Container started 35m Normal AddedInterface pod/test-curl-6985-eb56da Add eth0 [10.133.0.100/23] from ovn-kubernetes 35m Normal Created pod/test-curl-6985-eb56da Container created 35m Normal Killing pod/test-curl-6985-e9d00c Stopping container test-curl-6985-e9d00c 35m Normal Pulled pod/test-curl-6985-eb56da Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 35m Normal Pulled pod/test-curl-6985-3fd5e5 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/test-curl-6985-3fd5e5 Add eth0 [10.133.0.101/23] from ovn-kubernetes 35m Normal Started pod/test-curl-6985-3fd5e5 Container started 35m Normal Created pod/test-curl-6985-3fd5e5 Container created 35m Normal Killing pod/test-curl-6985-eb56da Stopping container test-curl-6985-eb56da 34m Normal Killing pod/test-curl-6985-3fd5e5 Stopping container test-curl-6985-3fd5e5 34m Normal Started pod/test-curl-6985-2630f8 Container started 34m Normal Created pod/test-curl-6985-2630f8 Container created 34m Normal Pulled pod/test-curl-6985-2630f8 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/test-curl-6985-2630f8 Add eth0 [10.133.0.102/23] from ovn-kubernetes 34m Normal Killing pod/test-curl-6985-2630f8 Stopping container test-curl-6985-2630f8 34m Normal AddedInterface pod/test-curl-6982-8bd389 Add eth0 [10.133.0.110/23] from ovn-kubernetes 34m Normal Pulled pod/test-curl-6982-8bd389 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 34m Normal Started pod/test-curl-6982-8bd389 Container started 34m Normal Created pod/test-curl-6982-8bd389 Container created 33m Normal Killing pod/test-curl-6982-8bd389 Stopping container test-curl-6982-8bd389 32m Normal Created pod/test-curl-6982-fb3e29 Container created 32m Normal Started pod/test-curl-6982-fb3e29 Container started 32m Normal Pulled pod/test-curl-6982-fb3e29 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/test-curl-6982-fb3e29 Add eth0 [10.133.0.117/23] from ovn-kubernetes 32m Normal Killing pod/test-curl-6982-fb3e29 Stopping container test-curl-6982-fb3e29 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Liveness probe failed: Get "https://10.132.0.47:8443/healthz": context deadline exceeded 29m Normal Killing pod/odh-model-controller-769875db56-8kjgc Stopping container manager 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Liveness probe failed: Get "https://10.132.0.47:8443/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Readiness probe failed: Get "https://10.132.0.47:8443/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Liveness probe failed: Get "https://10.132.0.47:8443/healthz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 29m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-fzvxh 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Readiness probe failed: Get "https://10.132.0.47:8443/readyz": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 29m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 29m Normal Pulling pod/odh-model-controller-769875db56-fzvxh Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 29m Normal AddedInterface pod/odh-model-controller-769875db56-fzvxh Add eth0 [10.134.0.152/23] from ovn-kubernetes 29m Normal Created pod/odh-model-controller-769875db56-fzvxh Container created 29m Normal Pulled pod/odh-model-controller-769875db56-fzvxh Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 494ms (494ms including waiting). Image size: 236755567 bytes. 29m Normal Started pod/odh-model-controller-769875db56-fzvxh Container started 29m Normal Killing pod/model-serving-api-75bd89fdfc-mdwfw Container server failed liveness probe, will be restarted 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Readiness probe failed: Get "https://10.132.0.47:8443/readyz": context deadline exceeded 29m Normal Pulled pod/model-serving-api-75bd89fdfc-mdwfw Container image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" already present on machine and can be accessed by the pod 29m Warning Unhealthy pod/model-serving-api-75bd89fdfc-mdwfw Readiness probe failed: Get "https://10.132.0.47:8443/readyz": dial tcp 10.132.0.47:8443: connect: connection refused 28m Normal Created pod/model-serving-api-75bd89fdfc-mdwfw Container created 28m Normal Started pod/model-serving-api-75bd89fdfc-mdwfw Container started 28m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-fzvxh_becbf8bb-e9e7-4888-8f5f-bb638369b974 became leader 17m Normal Killing pod/maas-controller-799454fcf4-24ntl Stopping container manager 17m Normal SuccessfulDelete replicaset/maas-controller-799454fcf4 Deleted pod: maas-controller-799454fcf4-24ntl 17m Normal SuccessfulCreate replicaset/maas-controller-799454fcf4 Created pod: maas-controller-799454fcf4-t4cpk 17m Normal AddedInterface pod/maas-controller-799454fcf4-t4cpk Add eth0 [10.134.0.167/23] from ovn-kubernetes 17m Normal Pulling pod/maas-controller-799454fcf4-t4cpk Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 17m Normal Pulled pod/maas-controller-799454fcf4-t4cpk Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 488ms (488ms including waiting). Image size: 200053271 bytes. 17m Normal Started pod/maas-controller-799454fcf4-t4cpk Container started 17m Normal Created pod/maas-controller-799454fcf4-t4cpk Container created 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-799454fcf4-t4cpk_aefacb8d-a687-4667-a55b-1dd1f917a1ab became leader 16m Normal Killing pod/maas-controller-799454fcf4-t4cpk Stopping container manager 16m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-799454fcf4 from 1 to 0 16m Normal SuccessfulDelete replicaset/maas-controller-799454fcf4 Deleted pod: maas-controller-799454fcf4-t4cpk 16m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-799454fcf4 from 0 to 1 16m Normal SuccessfulCreate replicaset/maas-controller-799454fcf4 Created pod: maas-controller-799454fcf4-2rncm 16m Normal Pulling pod/maas-controller-799454fcf4-2rncm Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 16m Normal AddedInterface pod/maas-controller-799454fcf4-2rncm Add eth0 [10.133.0.176/23] from ovn-kubernetes 16m Normal Pulled pod/maas-controller-799454fcf4-2rncm Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 1.62s (1.62s including waiting). Image size: 200053271 bytes. 16m Normal Started pod/maas-controller-799454fcf4-2rncm Container started 16m Normal Created pod/maas-controller-799454fcf4-2rncm Container created 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-799454fcf4-2rncm_ede3fbe8-a848-4848-9b72-5ba376c15ae7 became leader