=== events in opendatahub (2026-10-31T00:55:14Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 22m Normal Scheduled pod/maas-controller-7b8bb6f8d-hkrzz Successfully assigned opendatahub/maas-controller-7b8bb6f8d-hkrzz to ip-10-0-130-152.us-east-2.compute.internal 29m Normal Scheduled pod/test-curl-11085-e4ca10 Successfully assigned opendatahub/test-curl-11085-e4ca10 to ip-10-0-130-152.us-east-2.compute.internal 44m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-k9kkj Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-k9kkj to ip-10-0-134-160.us-east-2.compute.internal 32m Normal Scheduled pod/test-curl-11091-88ef39 Successfully assigned opendatahub/test-curl-11091-88ef39 to ip-10-0-130-152.us-east-2.compute.internal 55m Normal Scheduled pod/kserve-controller-manager-776cc66d47-jr6nm Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-jr6nm to ip-10-0-130-152.us-east-2.compute.internal 33m Normal Scheduled pod/test-curl-11091-3bb012 Successfully assigned opendatahub/test-curl-11091-3bb012 to ip-10-0-130-152.us-east-2.compute.internal 43m Normal Scheduled pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Successfully assigned opendatahub/ai-gateway-controller-6cf7bcbfb6-dbg69 to ip-10-0-143-16.us-east-2.compute.internal 45m Normal Scheduled pod/maas-controller-579cf7c6d-rdf75 Successfully assigned opendatahub/maas-controller-579cf7c6d-rdf75 to ip-10-0-143-16.us-east-2.compute.internal 22m Normal Scheduled pod/maas-controller-7b8bb6f8d-hb7vk Successfully assigned opendatahub/maas-controller-7b8bb6f8d-hb7vk to ip-10-0-130-152.us-east-2.compute.internal 43m Normal Scheduled pod/ai-gateway-controller-86f7d849b4-vgljn Successfully assigned opendatahub/ai-gateway-controller-86f7d849b4-vgljn to ip-10-0-143-16.us-east-2.compute.internal 45m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk to ip-10-0-130-152.us-east-2.compute.internal 55m Normal Scheduled pod/odh-model-controller-769875db56-mpwvn Successfully assigned opendatahub/odh-model-controller-769875db56-mpwvn to ip-10-0-130-152.us-east-2.compute.internal 36m Normal Scheduled pod/test-curl-11079-21bf41 Successfully assigned opendatahub/test-curl-11079-21bf41 to ip-10-0-143-16.us-east-2.compute.internal 36m Normal Scheduled pod/test-curl-11091-a1faed Successfully assigned opendatahub/test-curl-11091-a1faed to ip-10-0-134-160.us-east-2.compute.internal 45m Normal Scheduled pod/maas-controller-7b8bb6f8d-jrv29 Successfully assigned opendatahub/maas-controller-7b8bb6f8d-jrv29 to ip-10-0-143-16.us-east-2.compute.internal 27m Normal Scheduled pod/test-curl-11085-a749d3 Successfully assigned opendatahub/test-curl-11085-a749d3 to ip-10-0-130-152.us-east-2.compute.internal 55m Normal Scheduled pod/model-serving-api-75bd89fdfc-sxsw6 Successfully assigned opendatahub/model-serving-api-75bd89fdfc-sxsw6 to ip-10-0-130-152.us-east-2.compute.internal 56m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-mpl9v to ip-10-0-143-16.us-east-2.compute.internal 55m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-l985n Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-l985n to ip-10-0-134-160.us-east-2.compute.internal 56m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-mpl9v 56m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 56m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Add eth0 [10.132.0.37/23] from ovn-kubernetes 56m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 56m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.561s (2.561s including waiting). Image size: 195729000 bytes. 56m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-mpl9v_fd146199-6d91-4d0d-be64-74b4c4d05cab became leader 55m Normal Pulling pod/kserve-controller-manager-776cc66d47-jr6nm Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 55m Normal Pulling pod/odh-model-controller-769875db56-mpwvn Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 55m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-jr6nm Add eth0 [10.134.0.25/23] from ovn-kubernetes 55m Normal AddedInterface pod/model-serving-api-75bd89fdfc-sxsw6 Add eth0 [10.134.0.26/23] from ovn-kubernetes 55m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-l985n Add eth0 [10.133.0.31/23] from ovn-kubernetes 55m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-l985n Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 55m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-l985n 55m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-jr6nm 55m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 55m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 55m Normal Pulling pod/model-serving-api-75bd89fdfc-sxsw6 Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 55m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-mpwvn 55m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 55m Normal AddedInterface pod/odh-model-controller-769875db56-mpwvn Add eth0 [10.134.0.27/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-sxsw6 55m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 55m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-l985n Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.152s (2.152s including waiting). Image size: 235100981 bytes. 55m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-l985n Container created 55m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-l985n Container started 55m Normal Started pod/kserve-controller-manager-776cc66d47-jr6nm Container started 55m Normal Pulled pod/kserve-controller-manager-776cc66d47-jr6nm Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.915s (2.915s including waiting). Image size: 247676581 bytes. 55m Normal Created pod/kserve-controller-manager-776cc66d47-jr6nm Container created 55m Normal Created pod/model-serving-api-75bd89fdfc-sxsw6 Container created 55m Normal Started pod/model-serving-api-75bd89fdfc-sxsw6 Container started 55m Normal Pulled pod/model-serving-api-75bd89fdfc-sxsw6 Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.601s (2.601s including waiting). Image size: 184925761 bytes. 55m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-mpwvn_ab61e13a-686e-428c-a828-3ef0830eebce became leader 55m Normal Started pod/odh-model-controller-769875db56-mpwvn Container started 55m Normal Created pod/odh-model-controller-769875db56-mpwvn Container created 55m Normal Pulled pod/odh-model-controller-769875db56-mpwvn Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.426s (3.426s including waiting). Image size: 236755567 bytes. 55m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-l985n_3f7163c3-787e-45f1-9475-8a1d6c352bef became leader 55m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-jr6nm_be79bc93-4c98-437d-99dd-374c62b8d422 became leader 45m Normal SuccessfulCreate replicaset/maas-controller-579cf7c6d Created pod: maas-controller-579cf7c6d-rdf75 45m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-579cf7c6d from 0 to 1 45m Normal AddedInterface pod/maas-controller-579cf7c6d-rdf75 Add eth0 [10.132.0.41/23] from ovn-kubernetes 45m Normal AddedInterface pod/maas-controller-7b8bb6f8d-jrv29 Add eth0 [10.132.0.42/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/maas-controller-7b8bb6f8d Created pod: maas-controller-7b8bb6f8d-jrv29 45m Normal Pulling pod/maas-controller-579cf7c6d-rdf75 Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 45m Normal Pulled pod/maas-controller-7b8bb6f8d-jrv29 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 2.824s (2.824s including waiting). Image size: 200053271 bytes. 45m Normal Started pod/maas-controller-579cf7c6d-rdf75 Container started 45m Normal Created pod/maas-controller-579cf7c6d-rdf75 Container created 45m Normal Pulled pod/maas-controller-579cf7c6d-rdf75 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 2.976s (2.976s including waiting). Image size: 200053271 bytes. 45m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-579cf7c6d-rdf75_22f0ea0b-bc1f-4d19-be41-e814b03970b1 became leader 45m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-4w2nk 45m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk Add eth0 [10.134.0.32/23] from ovn-kubernetes 45m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 45m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 45m Normal SuccessfulDelete replicaset/maas-controller-579cf7c6d Deleted pod: maas-controller-579cf7c6d-rdf75 45m Normal Killing pod/maas-controller-579cf7c6d-rdf75 Stopping container manager 45m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-579cf7c6d from 1 to 0 45m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 19.132s (19.132s including waiting). Image size: 1089502687 bytes. 45m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk Container started 45m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-4w2nk Container created 44m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7b8bb6f8d-jrv29_e6a3abe9-827e-4972-ac3d-8c6314f504ee became leader 44m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-k9kkj 44m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 44m Normal Created pod/llmisvc-controller-manager-6466c554d8-k9kkj Container created 44m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-k9kkj Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 392ms (392ms including waiting). Image size: 235100981 bytes. 44m Normal Started pod/llmisvc-controller-manager-6466c554d8-k9kkj Container started 44m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-k9kkj Add eth0 [10.133.0.35/23] from ovn-kubernetes 44m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-k9kkj Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 44m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-l985n 44m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-l985n Stopping container manager 44m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 43m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-k9kkj_780182bf-3286-43d3-a35b-6105b6c08275 became leader 43m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-6cf7bcbfb6 from 0 to 1 43m Normal AddedInterface pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Add eth0 [10.132.0.54/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/ai-gateway-controller-6cf7bcbfb6 Created pod: ai-gateway-controller-6cf7bcbfb6-dbg69 43m Normal Pulling pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" 43m Normal Started pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Container started 43m Normal Pulled pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 2.549s (2.549s including waiting). Image size: 183987645 bytes. 43m Normal Created pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Container created 43m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-6cf7bcbfb6-dbg69_b023995c-9142-4bc0-acd9-cc5241cc3269 became leader 43m Normal SuccessfulCreate replicaset/ai-gateway-controller-86f7d849b4 Created pod: ai-gateway-controller-86f7d849b4-vgljn 43m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-86f7d849b4 from 0 to 1 43m Normal Pulled pod/ai-gateway-controller-86f7d849b4-vgljn Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 501ms (501ms including waiting). Image size: 183987645 bytes. 43m Normal AddedInterface pod/ai-gateway-controller-86f7d849b4-vgljn Add eth0 [10.132.0.56/23] from ovn-kubernetes 42m Normal SuccessfulDelete replicaset/ai-gateway-controller-6cf7bcbfb6 Deleted pod: ai-gateway-controller-6cf7bcbfb6-dbg69 42m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-6cf7bcbfb6 from 1 to 0 42m Normal Killing pod/ai-gateway-controller-6cf7bcbfb6-dbg69 Stopping container manager 42m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-86f7d849b4-vgljn_d112a155-603b-40c5-9db3-1b9c33b78fbc became leader 36m Normal AddedInterface pod/test-curl-11091-a1faed Add eth0 [10.133.0.80/23] from ovn-kubernetes 36m Normal Pulling pod/test-curl-11091-a1faed Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 36m Normal Pulling pod/test-curl-11079-21bf41 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 36m Normal AddedInterface pod/test-curl-11079-21bf41 Add eth0 [10.132.0.146/23] from ovn-kubernetes 36m Normal Created pod/test-curl-11079-21bf41 Container created 36m Normal Pulled pod/test-curl-11079-21bf41 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 4.47s (4.47s including waiting). Image size: 106100986 bytes. 36m Normal Created pod/test-curl-11091-a1faed Container created 36m Normal Started pod/test-curl-11091-a1faed Container started 36m Normal Started pod/test-curl-11079-21bf41 Container started 36m Normal Pulled pod/test-curl-11091-a1faed Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 4.24s (4.24s including waiting). Image size: 106100986 bytes. 36m Normal Killing pod/test-curl-11091-a1faed Stopping container test-curl-11091-a1faed 36m Normal Killing pod/test-curl-11079-21bf41 Stopping container test-curl-11079-21bf41 33m Normal AddedInterface pod/test-curl-11091-3bb012 Add eth0 [10.134.0.123/23] from ovn-kubernetes 33m Normal Pulling pod/test-curl-11091-3bb012 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 33m Normal Pulled pod/test-curl-11091-3bb012 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 1.04s (1.04s including waiting). Image size: 106100986 bytes. 33m Normal Started pod/test-curl-11091-3bb012 Container started 33m Normal Created pod/test-curl-11091-3bb012 Container created 32m Normal Killing pod/test-curl-11091-3bb012 Stopping container test-curl-11091-3bb012 32m Normal Created pod/test-curl-11091-88ef39 Container created 32m Normal Started pod/test-curl-11091-88ef39 Container started 32m Normal AddedInterface pod/test-curl-11091-88ef39 Add eth0 [10.134.0.124/23] from ovn-kubernetes 32m Normal Pulled pod/test-curl-11091-88ef39 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 32m Normal Killing pod/test-curl-11091-88ef39 Stopping container test-curl-11091-88ef39 32m Warning Unhealthy pod/ai-gateway-controller-86f7d849b4-vgljn Readiness probe failed: Get "http://10.132.0.56:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning Unhealthy pod/ai-gateway-controller-86f7d849b4-vgljn Liveness probe failed: Get "http://10.132.0.56:8081/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning Unhealthy pod/maas-controller-7b8bb6f8d-jrv29 Liveness probe failed: Get "http://10.132.0.42:8081/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Liveness probe failed: Get "http://10.132.0.37:8081/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 32m Normal Pulling pod/ai-gateway-controller-86f7d849b4-vgljn Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" 32m Normal TaintManagerEviction pod/maas-controller-7b8bb6f8d-jrv29 Cancelling deletion of Pod opendatahub/maas-controller-7b8bb6f8d-jrv29 32m Normal Created pod/ai-gateway-controller-86f7d849b4-vgljn Container created 32m Normal Started pod/ai-gateway-controller-86f7d849b4-vgljn Container started 32m Normal TaintManagerEviction pod/ai-gateway-controller-86f7d849b4-vgljn Cancelling deletion of Pod opendatahub/ai-gateway-controller-86f7d849b4-vgljn 32m Normal Pulled pod/ai-gateway-controller-86f7d849b4-vgljn Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-185" in 6.046s (6.046s including waiting). Image size: 183987645 bytes. 32m Normal TaintManagerEviction pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Cancelling deletion of Pod opendatahub/kserve-module-controller-manager-7566cf5fc8-mpl9v 31m Warning Unhealthy pod/maas-controller-7b8bb6f8d-jrv29 Readiness probe failed: Get "http://10.132.0.42:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Liveness probe failed: Get "http://10.132.0.37:8081/healthz": dial tcp 10.132.0.37:8081: connect: connection refused 31m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Readiness probe failed: Get "http://10.132.0.37:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-86f7d849b4-vgljn_c704cd3b-486a-4ea6-b0b7-759d13bba79d became leader 31m Warning Unhealthy pod/maas-controller-7b8bb6f8d-jrv29 Readiness probe failed: Get "http://10.132.0.42:8081/readyz": dial tcp 10.132.0.42:8081: connect: connection refused 31m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Readiness probe failed: Get "http://10.132.0.37:8081/readyz": dial tcp 10.132.0.37:8081: connect: connection refused 31m Warning Unhealthy pod/maas-controller-7b8bb6f8d-jrv29 Liveness probe failed: Get "http://10.132.0.42:8081/healthz": dial tcp 10.132.0.42:8081: connect: connection refused 31m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Container image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" already present on machine and can be accessed by the pod 31m Normal Pulling pod/maas-controller-7b8bb6f8d-jrv29 Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 31m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Container created 31m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-mpl9v Container started 31m Normal Pulled pod/maas-controller-7b8bb6f8d-jrv29 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 4.867s (4.867s including waiting). Image size: 200053271 bytes. 31m Normal Created pod/maas-controller-7b8bb6f8d-jrv29 Container created 31m Normal Started pod/maas-controller-7b8bb6f8d-jrv29 Container started 30m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7b8bb6f8d-jrv29_68f4b865-2a26-4cce-a585-35c6a5f0c88f became leader 30m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-mpl9v_e9d2f19f-135c-4e93-b3dc-84fd9f05051a became leader 29m Normal Started pod/test-curl-11085-e4ca10 Container started 29m Normal Created pod/test-curl-11085-e4ca10 Container created 29m Normal AddedInterface pod/test-curl-11085-e4ca10 Add eth0 [10.134.0.136/23] from ovn-kubernetes 29m Normal Pulled pod/test-curl-11085-e4ca10 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 29m Normal Killing pod/test-curl-11085-e4ca10 Stopping container test-curl-11085-e4ca10 27m Normal AddedInterface pod/test-curl-11085-a749d3 Add eth0 [10.134.0.149/23] from ovn-kubernetes 27m Normal Pulled pod/test-curl-11085-a749d3 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 27m Normal Created pod/test-curl-11085-a749d3 Container created 27m Normal Started pod/test-curl-11085-a749d3 Container started 27m Normal Killing pod/test-curl-11085-a749d3 Stopping container test-curl-11085-a749d3 23m Normal Killing pod/maas-controller-7b8bb6f8d-jrv29 Stopping container manager 23m Normal SuccessfulDelete replicaset/maas-controller-7b8bb6f8d Deleted pod: maas-controller-7b8bb6f8d-jrv29 22m Normal SuccessfulCreate replicaset/maas-controller-7b8bb6f8d Created pod: maas-controller-7b8bb6f8d-hkrzz 22m Normal Pulling pod/maas-controller-7b8bb6f8d-hkrzz Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 22m Normal AddedInterface pod/maas-controller-7b8bb6f8d-hkrzz Add eth0 [10.134.0.159/23] from ovn-kubernetes 22m Normal Started pod/maas-controller-7b8bb6f8d-hkrzz Container started 22m Normal Created pod/maas-controller-7b8bb6f8d-hkrzz Container created 22m Normal Pulled pod/maas-controller-7b8bb6f8d-hkrzz Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 2.676s (2.676s including waiting). Image size: 200053271 bytes. 22m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7b8bb6f8d-hkrzz_883a1903-289a-4094-b2b2-1f9b7323ce96 became leader 22m Normal Killing pod/maas-controller-7b8bb6f8d-hkrzz Stopping container manager 22m Normal SuccessfulDelete replicaset/maas-controller-7b8bb6f8d Deleted pod: maas-controller-7b8bb6f8d-hkrzz 22m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-7b8bb6f8d from 1 to 0 22m Normal AddedInterface pod/maas-controller-7b8bb6f8d-hb7vk Add eth0 [10.134.0.160/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/maas-controller-7b8bb6f8d Created pod: maas-controller-7b8bb6f8d-hb7vk 22m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-7b8bb6f8d from 0 to 1 22m Normal Started pod/maas-controller-7b8bb6f8d-hb7vk Container started 22m Normal Created pod/maas-controller-7b8bb6f8d-hb7vk Container created 22m Normal Pulled pod/maas-controller-7b8bb6f8d-hb7vk Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 429ms (429ms including waiting). Image size: 200053271 bytes. 22m Normal Pulling pod/maas-controller-7b8bb6f8d-hb7vk Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 22m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7b8bb6f8d-hb7vk_0e8712c9-8825-40a0-a2ef-e7e9be943bdd became leader