=== events in opendatahub (2026-10-31T00:20:46Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 45m Normal Scheduled pod/llmisvc-controller-manager-6466c554d8-qpd89 Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-qpd89 to ip-10-0-134-83.us-east-2.compute.internal 32m Normal Scheduled pod/test-curl-11079-7e86ce Successfully assigned opendatahub/test-curl-11079-7e86ce to ip-10-0-134-83.us-east-2.compute.internal 46m Normal Scheduled pod/maas-controller-7d4db8f8cf-ngxnw Successfully assigned opendatahub/maas-controller-7d4db8f8cf-ngxnw to ip-10-0-142-208.us-east-2.compute.internal 38m Normal Scheduled pod/test-curl-11073-865688 Successfully assigned opendatahub/test-curl-11073-865688 to ip-10-0-142-208.us-east-2.compute.internal 57m Normal Scheduled pod/kserve-module-controller-manager-7566cf5fc8-5n89j Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-5n89j to ip-10-0-137-81.us-east-2.compute.internal 31m Normal Scheduled pod/test-curl-11064-86556d Successfully assigned opendatahub/test-curl-11064-86556d to ip-10-0-134-83.us-east-2.compute.internal 44m Normal Scheduled pod/ai-gateway-controller-58f8946bc6-2bcnq Successfully assigned opendatahub/ai-gateway-controller-58f8946bc6-2bcnq to ip-10-0-142-208.us-east-2.compute.internal 44m Normal Scheduled pod/ai-gateway-controller-655fc879db-5vqfn Successfully assigned opendatahub/ai-gateway-controller-655fc879db-5vqfn to ip-10-0-142-208.us-east-2.compute.internal 31m Normal Scheduled pod/test-curl-11064-914b2c Successfully assigned opendatahub/test-curl-11064-914b2c to ip-10-0-134-83.us-east-2.compute.internal 31m Normal Scheduled pod/test-curl-11064-aa12de Successfully assigned opendatahub/test-curl-11064-aa12de to ip-10-0-134-83.us-east-2.compute.internal 16m Normal Scheduled pod/maas-controller-7d4db8f8cf-2l8pm Successfully assigned opendatahub/maas-controller-7d4db8f8cf-2l8pm to ip-10-0-142-208.us-east-2.compute.internal 46m Normal Scheduled pod/maas-controller-579cf7c6d-kbk66 Successfully assigned opendatahub/maas-controller-579cf7c6d-kbk66 to ip-10-0-142-208.us-east-2.compute.internal 56m Normal Scheduled pod/odh-model-controller-769875db56-4blbf Successfully assigned opendatahub/odh-model-controller-769875db56-4blbf to ip-10-0-137-81.us-east-2.compute.internal 46m Normal Scheduled pod/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 to ip-10-0-142-208.us-east-2.compute.internal 37m Normal Scheduled pod/test-curl-11073-f630c8 Successfully assigned opendatahub/test-curl-11073-f630c8 to ip-10-0-142-208.us-east-2.compute.internal 17m Normal Scheduled pod/maas-controller-7d4db8f8cf-tr6gw Successfully assigned opendatahub/maas-controller-7d4db8f8cf-tr6gw to ip-10-0-142-208.us-east-2.compute.internal 31m Normal Scheduled pod/test-curl-11064-5624c6 Successfully assigned opendatahub/test-curl-11064-5624c6 to ip-10-0-134-83.us-east-2.compute.internal 56m Normal Scheduled pod/kserve-controller-manager-776cc66d47-pm8s9 Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-pm8s9 to ip-10-0-134-83.us-east-2.compute.internal 56m Normal Scheduled pod/llmisvc-controller-manager-869d5dcc5d-46r5z Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-46r5z to ip-10-0-134-83.us-east-2.compute.internal 31m Normal Scheduled pod/test-curl-11079-163a14 Successfully assigned opendatahub/test-curl-11079-163a14 to ip-10-0-134-83.us-east-2.compute.internal 56m Normal Scheduled pod/model-serving-api-75bd89fdfc-8l59t Successfully assigned opendatahub/model-serving-api-75bd89fdfc-8l59t to ip-10-0-142-208.us-east-2.compute.internal 57m Normal SuccessfulCreate replicaset/kserve-module-controller-manager-7566cf5fc8 Created pod: kserve-module-controller-manager-7566cf5fc8-5n89j 57m Normal ScalingReplicaSet deployment/kserve-module-controller-manager Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1 57m Normal Pulling pod/kserve-module-controller-manager-7566cf5fc8-5n89j Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" 57m Normal AddedInterface pod/kserve-module-controller-manager-7566cf5fc8-5n89j Add eth0 [10.134.0.23/23] from ovn-kubernetes 57m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-5n89j Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.408s (2.408s including waiting). Image size: 195729000 bytes. 57m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-5n89j_4e3af7a2-227b-43f4-a107-d4a6a6ed2ab5 became leader 56m Normal SuccessfulCreate replicaset/odh-model-controller-769875db56 Created pod: odh-model-controller-769875db56-4blbf 56m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-869d5dcc5d Created pod: llmisvc-controller-manager-869d5dcc5d-46r5z 56m Normal ScalingReplicaSet deployment/kserve-controller-manager Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1 56m Normal SuccessfulCreate replicaset/kserve-controller-manager-776cc66d47 Created pod: kserve-controller-manager-776cc66d47-pm8s9 56m Normal ScalingReplicaSet deployment/model-serving-api Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1 56m Normal SuccessfulCreate replicaset/model-serving-api-75bd89fdfc Created pod: model-serving-api-75bd89fdfc-8l59t 56m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1 56m Normal ScalingReplicaSet deployment/odh-model-controller Scaled up replica set odh-model-controller-769875db56 from 0 to 1 56m Normal AddedInterface pod/llmisvc-controller-manager-869d5dcc5d-46r5z Add eth0 [10.133.0.40/23] from ovn-kubernetes 56m Normal Pulling pod/model-serving-api-75bd89fdfc-8l59t Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" 56m Normal Pulling pod/llmisvc-controller-manager-869d5dcc5d-46r5z Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 56m Normal AddedInterface pod/model-serving-api-75bd89fdfc-8l59t Add eth0 [10.132.0.28/23] from ovn-kubernetes 56m Normal AddedInterface pod/odh-model-controller-769875db56-4blbf Add eth0 [10.134.0.25/23] from ovn-kubernetes 56m Normal AddedInterface pod/kserve-controller-manager-776cc66d47-pm8s9 Add eth0 [10.133.0.39/23] from ovn-kubernetes 56m Normal Pulling pod/kserve-controller-manager-776cc66d47-pm8s9 Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" 56m Normal Pulled pod/model-serving-api-75bd89fdfc-8l59t Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 1.792s (1.792s including waiting). Image size: 184925761 bytes. 56m Normal Started pod/model-serving-api-75bd89fdfc-8l59t Container started 56m Normal Created pod/model-serving-api-75bd89fdfc-8l59t Container created 56m Normal Started pod/llmisvc-controller-manager-869d5dcc5d-46r5z Container started 56m Normal Created pod/llmisvc-controller-manager-869d5dcc5d-46r5z Container created 56m Normal Pulled pod/llmisvc-controller-manager-869d5dcc5d-46r5z Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.19s (3.191s including waiting). Image size: 235100981 bytes. 56m Normal Pulled pod/odh-model-controller-769875db56-4blbf Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 2.623s (2.623s including waiting). Image size: 236755567 bytes. 56m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-4blbf_a14ba428-054a-4c50-bb9d-8c8a8c8a3fdf became leader 56m Normal Created pod/kserve-controller-manager-776cc66d47-pm8s9 Container created 56m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-869d5dcc5d-46r5z_58606dcf-fa17-48cf-9d0e-3058ccfc58c3 became leader 56m Normal Pulled pod/kserve-controller-manager-776cc66d47-pm8s9 Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.738s (3.738s including waiting). Image size: 247676581 bytes. 56m Normal Started pod/kserve-controller-manager-776cc66d47-pm8s9 Container started 56m Normal LeaderElection lease/kserve-controller-manager-leader-lock kserve-controller-manager-776cc66d47-pm8s9_5351ab21-9e56-4b47-a44b-b833e3253789 became leader 46m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-579cf7c6d from 0 to 1 46m Normal SuccessfulCreate replicaset/maas-controller-579cf7c6d Created pod: maas-controller-579cf7c6d-kbk66 46m Normal Pulling pod/maas-controller-7d4db8f8cf-ngxnw Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 46m Normal AddedInterface pod/maas-controller-579cf7c6d-kbk66 Add eth0 [10.132.0.33/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/maas-controller-7d4db8f8cf Created pod: maas-controller-7d4db8f8cf-ngxnw 46m Normal AddedInterface pod/maas-controller-7d4db8f8cf-ngxnw Add eth0 [10.132.0.34/23] from ovn-kubernetes 46m Normal Pulling pod/maas-controller-579cf7c6d-kbk66 Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 46m Normal Started pod/maas-controller-579cf7c6d-kbk66 Container started 46m Normal Created pod/maas-controller-7d4db8f8cf-ngxnw Container created 46m Normal Started pod/maas-controller-7d4db8f8cf-ngxnw Container started 46m Normal Pulled pod/maas-controller-7d4db8f8cf-ngxnw Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 2.817s (2.817s including waiting). Image size: 200053271 bytes. 46m Normal Created pod/maas-controller-579cf7c6d-kbk66 Container created 46m Normal Pulled pod/maas-controller-579cf7c6d-kbk66 Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 2.983s (2.983s including waiting). Image size: 200053271 bytes. 46m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7d4db8f8cf-ngxnw_1f76cafd-eba2-4bd6-89b3-bf65e012d36c became leader 46m Normal ScalingReplicaSet deployment/usage-logs-tenancy-proxy Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1 46m Normal AddedInterface pod/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 Add eth0 [10.132.0.35/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/usage-logs-tenancy-proxy-9c5d49bcd Created pod: usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 46m Normal Pulling pod/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" 46m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-579cf7c6d from 1 to 0 46m Normal Killing pod/maas-controller-579cf7c6d-kbk66 Stopping container manager 46m Normal SuccessfulDelete replicaset/maas-controller-579cf7c6d Deleted pod: maas-controller-579cf7c6d-kbk66 46m Normal Started pod/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 Container started 46m Normal Pulled pod/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 15.743s (15.743s including waiting). Image size: 1089502687 bytes. 46m Normal Created pod/usage-logs-tenancy-proxy-9c5d49bcd-r2bg5 Container created 45m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1 45m Normal SuccessfulCreate replicaset/llmisvc-controller-manager-6466c554d8 Created pod: llmisvc-controller-manager-6466c554d8-qpd89 45m Normal Pulled pod/llmisvc-controller-manager-6466c554d8-qpd89 Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 394ms (394ms including waiting). Image size: 235100981 bytes. 45m Normal AddedInterface pod/llmisvc-controller-manager-6466c554d8-qpd89 Add eth0 [10.133.0.45/23] from ovn-kubernetes 45m Normal Pulling pod/llmisvc-controller-manager-6466c554d8-qpd89 Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" 45m Normal Started pod/llmisvc-controller-manager-6466c554d8-qpd89 Container started 45m Normal Created pod/llmisvc-controller-manager-6466c554d8-qpd89 Container created 45m Normal ScalingReplicaSet deployment/llmisvc-controller-manager Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0 45m Normal SuccessfulDelete replicaset/llmisvc-controller-manager-869d5dcc5d Deleted pod: llmisvc-controller-manager-869d5dcc5d-46r5z 45m Normal Killing pod/llmisvc-controller-manager-869d5dcc5d-46r5z Stopping container manager 44m Normal LeaderElection lease/llminferenceservice-kserve-controller-manager llmisvc-controller-manager-6466c554d8-qpd89_216b1725-6df3-4081-bfc8-d5ece210fa91 became leader 44m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-655fc879db from 0 to 1 44m Normal SuccessfulCreate replicaset/ai-gateway-controller-655fc879db Created pod: ai-gateway-controller-655fc879db-5vqfn 44m Normal Pulling pod/ai-gateway-controller-655fc879db-5vqfn Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-152" 44m Normal AddedInterface pod/ai-gateway-controller-655fc879db-5vqfn Add eth0 [10.132.0.48/23] from ovn-kubernetes 44m Normal Created pod/ai-gateway-controller-655fc879db-5vqfn Container created 44m Normal Started pod/ai-gateway-controller-655fc879db-5vqfn Container started 44m Normal Pulled pod/ai-gateway-controller-655fc879db-5vqfn Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-152" in 4.044s (4.044s including waiting). Image size: 183979410 bytes. 44m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-655fc879db-5vqfn_8dd42546-847d-44ef-9b9e-4a7f67433318 became leader 44m Normal SuccessfulCreate replicaset/ai-gateway-controller-58f8946bc6 Created pod: ai-gateway-controller-58f8946bc6-2bcnq 44m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled up replica set ai-gateway-controller-58f8946bc6 from 0 to 1 44m Normal Pulling pod/ai-gateway-controller-58f8946bc6-2bcnq Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-152" 44m Normal Pulled pod/ai-gateway-controller-58f8946bc6-2bcnq Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:odh-pr-152" in 378ms (378ms including waiting). Image size: 183979410 bytes. 44m Normal Created pod/ai-gateway-controller-58f8946bc6-2bcnq Container created 44m Normal Started pod/ai-gateway-controller-58f8946bc6-2bcnq Container started 44m Normal AddedInterface pod/ai-gateway-controller-58f8946bc6-2bcnq Add eth0 [10.132.0.51/23] from ovn-kubernetes 43m Normal Killing pod/ai-gateway-controller-655fc879db-5vqfn Stopping container manager 43m Normal ScalingReplicaSet deployment/ai-gateway-controller Scaled down replica set ai-gateway-controller-655fc879db from 1 to 0 43m Normal SuccessfulDelete replicaset/ai-gateway-controller-655fc879db Deleted pod: ai-gateway-controller-655fc879db-5vqfn 43m Normal LeaderElection lease/ai-gateway-controller-leader.opendatahub.io ai-gateway-controller-58f8946bc6-2bcnq_e55e9ce5-1558-4f61-a61d-a9308d92c677 became leader 38m Normal AddedInterface pod/test-curl-11073-865688 Add eth0 [10.132.0.94/23] from ovn-kubernetes 38m Normal Pulling pod/test-curl-11073-865688 Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 37m Normal Pulled pod/test-curl-11073-865688 Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 3.298s (3.298s including waiting). Image size: 106100986 bytes. 37m Normal Created pod/test-curl-11073-865688 Container created 37m Normal Started pod/test-curl-11073-865688 Container started 37m Normal Created pod/test-curl-11073-f630c8 Container created 37m Normal AddedInterface pod/test-curl-11073-f630c8 Add eth0 [10.132.0.95/23] from ovn-kubernetes 37m Normal Started pod/test-curl-11073-f630c8 Container started 37m Normal Killing pod/test-curl-11073-865688 Stopping container test-curl-11073-865688 37m Normal Pulled pod/test-curl-11073-f630c8 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 37m Normal Killing pod/test-curl-11073-f630c8 Stopping container test-curl-11073-f630c8 32m Normal Pulling pod/test-curl-11079-7e86ce Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" 32m Normal AddedInterface pod/test-curl-11079-7e86ce Add eth0 [10.133.0.123/23] from ovn-kubernetes 32m Normal Created pod/test-curl-11079-7e86ce Container created 32m Normal Pulled pod/test-curl-11079-7e86ce Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 4.545s (4.545s including waiting). Image size: 106100986 bytes. 32m Normal Started pod/test-curl-11079-7e86ce Container started 32m Normal Killing pod/test-curl-11079-7e86ce Stopping container test-curl-11079-7e86ce 31m Normal Pulled pod/test-curl-11064-aa12de Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/test-curl-11064-aa12de Add eth0 [10.133.0.135/23] from ovn-kubernetes 31m Normal Started pod/test-curl-11064-aa12de Container started 31m Normal Created pod/test-curl-11064-aa12de Container created 31m Normal Created pod/test-curl-11064-5624c6 Container created 31m Normal AddedInterface pod/test-curl-11064-5624c6 Add eth0 [10.133.0.136/23] from ovn-kubernetes 31m Normal Started pod/test-curl-11064-5624c6 Container started 31m Normal Killing pod/test-curl-11064-aa12de Stopping container test-curl-11064-aa12de 31m Normal Pulled pod/test-curl-11064-5624c6 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 31m Normal Started pod/test-curl-11064-86556d Container started 31m Normal AddedInterface pod/test-curl-11064-86556d Add eth0 [10.133.0.137/23] from ovn-kubernetes 31m Normal Created pod/test-curl-11064-86556d Container created 31m Normal Pulled pod/test-curl-11064-86556d Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 31m Normal Killing pod/test-curl-11064-5624c6 Stopping container test-curl-11064-5624c6 31m Normal Started pod/test-curl-11064-914b2c Container started 31m Normal AddedInterface pod/test-curl-11064-914b2c Add eth0 [10.133.0.139/23] from ovn-kubernetes 31m Normal Created pod/test-curl-11064-914b2c Container created 31m Normal Pulled pod/test-curl-11064-914b2c Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 31m Normal Killing pod/test-curl-11064-86556d Stopping container test-curl-11064-86556d 31m Normal Killing pod/test-curl-11064-914b2c Stopping container test-curl-11064-914b2c 31m Normal Pulled pod/test-curl-11079-163a14 Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/test-curl-11079-163a14 Add eth0 [10.133.0.140/23] from ovn-kubernetes 31m Normal Started pod/test-curl-11079-163a14 Container started 31m Normal Created pod/test-curl-11079-163a14 Container created 31m Normal Killing pod/test-curl-11079-163a14 Stopping container test-curl-11079-163a14 27m Warning Unhealthy pod/odh-model-controller-769875db56-4blbf Readiness probe failed: Get "http://10.134.0.25:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning NodeNotReady pod/odh-model-controller-769875db56-4blbf Node is not ready 26m Warning NodeNotReady pod/kserve-module-controller-manager-7566cf5fc8-5n89j Node is not ready 25m Warning Unhealthy pod/odh-model-controller-769875db56-4blbf Liveness probe failed: Get "http://10.134.0.25:8081/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-5n89j Readiness probe failed: Get "http://10.134.0.23:8081/readyz": dial tcp 10.134.0.23:8081: connect: connection refused 23m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-5n89j Liveness probe failed: Get "http://10.134.0.23:8081/healthz": dial tcp 10.134.0.23:8081: connect: connection refused 23m Normal Pulling pod/odh-model-controller-769875db56-4blbf Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" 23m Normal Killing pod/odh-model-controller-769875db56-4blbf Container manager failed liveness probe, will be restarted 23m Normal Pulled pod/kserve-module-controller-manager-7566cf5fc8-5n89j Container image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" already present on machine and can be accessed by the pod 22m Normal Created pod/kserve-module-controller-manager-7566cf5fc8-5n89j Container created 22m Normal Started pod/kserve-module-controller-manager-7566cf5fc8-5n89j Container started 22m Normal TaintManagerEviction pod/odh-model-controller-769875db56-4blbf Cancelling deletion of Pod opendatahub/odh-model-controller-769875db56-4blbf 22m Normal TaintManagerEviction pod/kserve-module-controller-manager-7566cf5fc8-5n89j Cancelling deletion of Pod opendatahub/kserve-module-controller-manager-7566cf5fc8-5n89j 22m Warning Unhealthy pod/odh-model-controller-769875db56-4blbf Readiness probe failed: Get "http://10.134.0.25:8081/readyz": dial tcp 10.134.0.25:8081: connect: connection refused 22m Normal Pulled pod/odh-model-controller-769875db56-4blbf Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 43.379s (43.379s including waiting). Image size: 236755567 bytes. 22m Normal Created pod/odh-model-controller-769875db56-4blbf Container created 22m Normal Started pod/odh-model-controller-769875db56-4blbf Container started 22m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-5n89j Liveness probe failed: Get "http://10.134.0.23:8081/healthz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Warning Unhealthy pod/kserve-module-controller-manager-7566cf5fc8-5n89j Readiness probe failed: Get "http://10.134.0.23:8081/readyz": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Normal LeaderElection lease/odh-model-controller.opendatahub.io odh-model-controller-769875db56-4blbf_4bac205d-899b-4057-8fad-bfd859f1872b became leader 21m Normal LeaderElection lease/kserve-module-controller-manager kserve-module-controller-manager-7566cf5fc8-5n89j_349f630d-c85b-4023-95d7-15cdf2b5db1a became leader 17m Normal SuccessfulDelete replicaset/maas-controller-7d4db8f8cf Deleted pod: maas-controller-7d4db8f8cf-ngxnw 17m Normal Killing pod/maas-controller-7d4db8f8cf-ngxnw Stopping container manager 17m Normal SuccessfulCreate replicaset/maas-controller-7d4db8f8cf Created pod: maas-controller-7d4db8f8cf-tr6gw 17m Normal Pulling pod/maas-controller-7d4db8f8cf-tr6gw Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 17m Normal AddedInterface pod/maas-controller-7d4db8f8cf-tr6gw Add eth0 [10.132.0.165/23] from ovn-kubernetes 17m Normal Started pod/maas-controller-7d4db8f8cf-tr6gw Container started 17m Normal Pulled pod/maas-controller-7d4db8f8cf-tr6gw Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 807ms (807ms including waiting). Image size: 200053271 bytes. 17m Normal Created pod/maas-controller-7d4db8f8cf-tr6gw Container created 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7d4db8f8cf-tr6gw_f8e37c52-462a-49a9-919d-cf3439b29a49 became leader 16m Normal ScalingReplicaSet deployment/maas-controller Scaled down replica set maas-controller-7d4db8f8cf from 1 to 0 16m Normal SuccessfulDelete replicaset/maas-controller-7d4db8f8cf Deleted pod: maas-controller-7d4db8f8cf-tr6gw 16m Normal Killing pod/maas-controller-7d4db8f8cf-tr6gw Stopping container manager 16m Normal SuccessfulCreate replicaset/maas-controller-7d4db8f8cf Created pod: maas-controller-7d4db8f8cf-2l8pm 16m Normal ScalingReplicaSet deployment/maas-controller Scaled up replica set maas-controller-7d4db8f8cf from 0 to 1 16m Normal AddedInterface pod/maas-controller-7d4db8f8cf-2l8pm Add eth0 [10.132.0.167/23] from ovn-kubernetes 16m Normal Pulling pod/maas-controller-7d4db8f8cf-2l8pm Pulling image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" 16m Normal Pulled pod/maas-controller-7d4db8f8cf-2l8pm Successfully pulled image "quay.io/opendatahub/maas-controller:7879675aa66d002425c7b32375d6d0156eaa0c56" in 413ms (413ms including waiting). Image size: 200053271 bytes. 16m Normal Created pod/maas-controller-7d4db8f8cf-2l8pm Container created 16m Normal Started pod/maas-controller-7d4db8f8cf-2l8pm Container started 16m Normal LeaderElection lease/maas-controller.models-as-a-service.opendatahub.io maas-controller-7d4db8f8cf-2l8pm_08a2e179-cf0f-446e-aa48-8af8256f1d2e became leader