=== events in llm (2026-11-04T01:52:18Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 6m18s Normal Scheduled pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Successfully assigned llm/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l to ip-10-0-136-96.us-east-2.compute.internal 41m Normal Scheduled pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz to ip-10-0-136-96.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 to ip-10-0-140-186.us-east-2.compute.internal 6m3s Warning ModelNameConflict maasmodelref/e2e-same-id-a-f3467868 Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-b-f3467868 6m3s Warning ModelNameConflict maasmodelref/e2e-same-id-b-f3467868 Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-a-f3467868 41m Normal Scheduled pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk to ip-10-0-136-96.us-east-2.compute.internal 41m Normal Scheduled pod/echo-simulator-kserve-64b6f8f96c-pqxmd Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-pqxmd to ip-10-0-140-186.us-east-2.compute.internal 41m Normal Scheduled pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 to ip-10-0-136-96.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 to ip-10-0-136-96.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 to ip-10-0-136-96.us-east-2.compute.internal 5m51s Normal ModelNameConflictResolved maasmodelref/e2e-same-id-b-f3467868 Model identity is no longer shared with any other MaaSModelRef in this namespace 41m Normal Scheduled pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-k42tl to ip-10-0-136-96.us-east-2.compute.internal 6m34s Normal Scheduled pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Successfully assigned llm/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m to ip-10-0-136-96.us-east-2.compute.internal 41m Normal SuccessfulCreate replicaset/facebook-opt-125m-simulated-kserve-84b4ff656c Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.Deployment llm/facebook-opt-125m-simulated-kserve 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs 41m Normal ScalingReplicaSet deployment/facebook-opt-125m-simulated-kserve Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.Deployment llm/premium-simulated-simulated-premium-kserve 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route 41m Normal Created llminferenceservice/facebook-opt-125m-simulated Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc 41m Normal AddedInterface pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Add eth0 [10.133.0.29/23] from ovn-kubernetes 41m Normal Pulling pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal SuccessfulCreate replicaset/premium-simulated-simulated-premium-kserve-5f5c4b459f Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz 41m Normal ScalingReplicaSet deployment/premium-simulated-simulated-premium-kserve Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine 41m Normal Pulling pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal AddedInterface pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Add eth0 [10.133.0.30/23] from ovn-kubernetes 41m Normal Created llminferenceservice/premium-simulated-simulated-premium Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping 41m Normal Created pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Container created 41m Normal Created pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Container created 41m Normal Pulled pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.08s (1.08s including waiting). Image size: 98346788 bytes. 41m Normal Started pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Container started 41m Normal Started pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Container started 41m Normal Pulled pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.596s (1.596s including waiting). Image size: 98346788 bytes. 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default 41m Normal SuccessfulCreate replicaset/e2e-distinct-2-simulated-kserve-cbd4f5586 Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 41m Normal ScalingReplicaSet deployment/e2e-distinct-2-simulated-kserve Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1 41m Normal AddedInterface pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Add eth0 [10.133.0.31/23] from ovn-kubernetes 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc 41m Normal Pulling pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine 41m Normal Created llminferenceservice/e2e-distinct-2-simulated Created v1.Deployment llm/e2e-distinct-2-simulated-kserve 41m Normal Started pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Container started 41m Normal Pulled pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 308ms (308ms including waiting). Image size: 98346788 bytes. 41m Normal Created pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Container created 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs 41m Normal SuccessfulCreate replicaset/e2e-distinct-simulated-kserve-7b998b79d9 Created pod: e2e-distinct-simulated-kserve-7b998b79d9-k42tl 41m Normal ScalingReplicaSet deployment/e2e-distinct-simulated-kserve Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.Deployment llm/e2e-distinct-simulated-kserve 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine 41m Normal Created llminferenceservice/e2e-distinct-simulated Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping 41m Normal SuccessfulCreate replicaset/e2e-embedding-simulated-kserve-67bf594cd6 Created pod: e2e-embedding-simulated-kserve-67bf594cd6-ghv26 41m Normal Pulling pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal AddedInterface pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Add eth0 [10.133.0.32/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/e2e-embedding-simulated-kserve Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.Deployment llm/e2e-embedding-simulated-kserve 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default 41m Normal Created llminferenceservice/e2e-embedding-simulated Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine 41m Normal Started pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Container started 41m Normal Created pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Container created 41m Normal Pulled pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 267ms (267ms including waiting). Image size: 98346788 bytes. 41m Normal Pulling pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal Started pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Container started 41m Normal AddedInterface pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Add eth0 [10.133.0.33/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 293ms (293ms including waiting). Image size: 98346788 bytes. 41m Normal Created pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Container created 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc 41m Normal ScalingReplicaSet deployment/e2e-trlp-test-simulated-kserve Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-trlp-test-simulated-kserve-69b588897d Created pod: e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.Deployment llm/e2e-trlp-test-simulated-kserve 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route 41m Normal Created llminferenceservice/e2e-trlp-test-simulated Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default 41m Normal SuccessfulCreate replicaset/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine 41m Normal AddedInterface pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Add eth0 [10.134.0.36/23] from ovn-kubernetes 41m Normal Pulling pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route 41m Normal ScalingReplicaSet deployment/e2e-unconfigured-facebook-opt-125m-simulated-kserve Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs 41m Normal Created llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve 41m Normal Started pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Container started 41m Normal Pulled pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.212s (1.212s including waiting). Image size: 98346788 bytes. 41m Normal Started pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Container started 41m Normal Created pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Container created 41m Normal Pulled pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 284ms (284ms including waiting). Image size: 98346788 bytes. 41m Normal Pulling pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" 41m Normal AddedInterface pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Add eth0 [10.133.0.34/23] from ovn-kubernetes 41m Normal Created pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Container created 41m Normal Created llminferenceservice/echo-simulator Created v1.Deployment llm/echo-simulator-kserve 41m Normal Created llminferenceservice/echo-simulator Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine 41m Normal Created llminferenceservice/echo-simulator Created v1.HTTPRoute llm/echo-simulator-kserve-route 41m Normal Created llminferenceservice/echo-simulator Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc 41m Normal SuccessfulCreate replicaset/echo-simulator-kserve-64b6f8f96c Created pod: echo-simulator-kserve-64b6f8f96c-pqxmd 41m Normal ScalingReplicaSet deployment/echo-simulator-kserve Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1 41m Normal Created llminferenceservice/echo-simulator Created v1.Service llm/echo-simulator-kserve-workload-svc 41m Normal Created llminferenceservice/echo-simulator Created v1.Secret llm/echo-simulator-kserve-self-signed-certs 41m Normal Created llminferenceservice/echo-simulator Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default 41m Normal AddedInterface pod/echo-simulator-kserve-64b6f8f96c-pqxmd Add eth0 [10.134.0.37/23] from ovn-kubernetes 41m Normal Pulling pod/echo-simulator-kserve-64b6f8f96c-pqxmd Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" 41m Normal Created llminferenceservice/echo-simulator Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping 41m Normal Pulled pod/echo-simulator-kserve-64b6f8f96c-pqxmd Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 1.884s (1.884s including waiting). Image size: 78208878 bytes. 41m Normal Created pod/echo-simulator-kserve-64b6f8f96c-pqxmd Container created 41m Normal LLMInferenceServiceReady llminferenceservice/facebook-opt-125m-simulated LLMInferenceService [facebook-opt-125m-simulated] is Ready 41m Normal Updated llminferenceservice/facebook-opt-125m-simulated Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs 41m Normal Started pod/echo-simulator-kserve-64b6f8f96c-pqxmd Container started 41m Normal Updated llminferenceservice/premium-simulated-simulated-premium Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs 41m Normal LLMInferenceServiceReady llminferenceservice/e2e-distinct-2-simulated LLMInferenceService [e2e-distinct-2-simulated] is Ready 41m Normal Updated llminferenceservice/e2e-distinct-2-simulated Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs 41m Normal Updated llminferenceservice/e2e-distinct-simulated Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs 41m Normal LLMInferenceServiceReady llminferenceservice/e2e-distinct-simulated LLMInferenceService [e2e-distinct-simulated] is Ready 41m Normal Updated llminferenceservice/e2e-embedding-simulated Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs 41m Normal LLMInferenceServiceReady llminferenceservice/e2e-embedding-simulated LLMInferenceService [e2e-embedding-simulated] is Ready 41m Normal LLMInferenceServiceReady llminferenceservice/e2e-trlp-test-simulated LLMInferenceService [e2e-trlp-test-simulated] is Ready 41m Normal Updated llminferenceservice/e2e-trlp-test-simulated Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs 41m Normal Updated llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs 41m Normal LLMInferenceServiceReady llminferenceservice/e2e-unconfigured-facebook-opt-125m-simulated LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready 41m Normal Updated llminferenceservice/echo-simulator Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs 35m Warning Unhealthy pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Readiness probe failed: Get "https://10.133.0.34:8000/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 34m Warning Unhealthy pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Liveness probe failed: Get "https://10.133.0.30:8000/health": context deadline exceeded 34m Warning Unhealthy pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Readiness probe failed: Get "https://10.133.0.32:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 34m Warning Unhealthy pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Liveness probe failed: Get "https://10.133.0.33:8000/health": context deadline exceeded 33m Warning Unhealthy pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Liveness probe failed: Get "https://10.133.0.29:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Readiness probe failed: Get "https://10.133.0.31:8000/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Readiness probe failed: Get "https://10.133.0.29:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 32m Warning Unhealthy pod/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8xgsvk Readiness probe failed: Get "https://10.133.0.34:8000/ready": context deadline exceeded 32m Warning Unhealthy pod/e2e-distinct-simulated-kserve-7b998b79d9-k42tl Readiness probe failed: Get "https://10.133.0.32:8000/ready": context deadline exceeded 32m Warning Unhealthy pod/e2e-distinct-2-simulated-kserve-cbd4f5586-vbvc9 Readiness probe failed: Get "https://10.133.0.31:8000/ready": context deadline exceeded 32m Warning Unhealthy pod/e2e-embedding-simulated-kserve-67bf594cd6-ghv26 Readiness probe failed: Get "https://10.133.0.33:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 32m Warning Unhealthy pod/facebook-opt-125m-simulated-kserve-84b4ff656c-f7bp8 Readiness probe failed: Get "https://10.133.0.29:8000/ready": context deadline exceeded 27m Warning Unhealthy pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Readiness probe failed: Get "https://10.134.0.36:8000/ready": context deadline exceeded 27m Warning Unhealthy pod/echo-simulator-kserve-64b6f8f96c-pqxmd Liveness probe failed: Get "https://10.134.0.37:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/echo-simulator-kserve-64b6f8f96c-pqxmd Liveness probe failed: Get "https://10.134.0.37:8000/health": context deadline exceeded 26m Warning Unhealthy pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Liveness probe failed: Get "https://10.134.0.36:8000/health": context deadline exceeded 26m Warning Unhealthy pod/echo-simulator-kserve-64b6f8f96c-pqxmd Liveness probe failed: Get "https://10.134.0.37:8000/health": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/echo-simulator-kserve-64b6f8f96c-pqxmd Readiness probe failed: Get "https://10.134.0.37:8000/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/echo-simulator-kserve-64b6f8f96c-pqxmd Readiness probe failed: Get "https://10.134.0.37:8000/ready": context deadline exceeded 26m Warning Unhealthy pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Readiness probe failed: Get "https://10.134.0.36:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/echo-simulator-kserve-64b6f8f96c-pqxmd Readiness probe failed: Get "https://10.134.0.37:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Readiness probe failed: Get "https://10.134.0.36:8000/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Liveness probe failed: Get "https://10.134.0.36:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 26m Normal TaintManagerEviction pod/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 Cancelling deletion of Pod llm/e2e-trlp-test-simulated-kserve-69b588897d-qb7m9 26m Normal TaintManagerEviction pod/echo-simulator-kserve-64b6f8f96c-pqxmd Cancelling deletion of Pod llm/echo-simulator-kserve-64b6f8f96c-pqxmd 25m Warning Error llminferenceservice/echo-simulator Reconciliation failed: failed to dry-run validate rendered config: Internal error occurred: failed calling webhook "connection-llmisvc-v1alpha2.odh-model-controller.opendatahub.io": failed to call webhook: Post "https://odh-model-controller-webhook-service.opendatahub.svc:443/mutate-serving-kserve-io-v1alpha2-llminferenceservice?timeout=10s": http: server gave HTTP response to HTTPS client 25m Warning Error llminferenceservice/echo-simulator Reconciliation failed: failed to dry-run validate rendered config: Internal error occurred: failed calling webhook "connection-llmisvc-v1alpha2.odh-model-controller.opendatahub.io": failed to call webhook: Post "https://odh-model-controller-webhook-service.opendatahub.svc:443/mutate-serving-kserve-io-v1alpha2-llminferenceservice?timeout=10s": no endpoints available for service "odh-model-controller-webhook-service" 25m Normal LLMInferenceServiceReady llminferenceservice/echo-simulator LLMInferenceService [echo-simulator] is Ready 24m Warning Unhealthy pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Readiness probe failed: Get "https://10.133.0.30:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/premium-simulated-simulated-premium-kserve-5f5c4b459f-rgsrz Readiness probe failed: Get "https://10.133.0.30:8000/ready": context deadline exceeded 24m Warning LLMInferenceServiceNotReady llminferenceservice/premium-simulated-simulated-premium LLMInferenceService [premium-simulated-simulated-premium] is no longer Ready because of: MainWorkloadReady, WorkloadsReady 24m Normal LLMInferenceServiceReady llminferenceservice/premium-simulated-simulated-premium LLMInferenceService [premium-simulated-simulated-premium] is Ready 6m35s Normal ScalingReplicaSet deployment/e2e-same-id-a-f3467868-kserve Scaled up replica set e2e-same-id-a-f3467868-kserve-6ff68659d9 from 0 to 1 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.Service llm/e2e-same-id-a-f3467868-kserve-workload-svc 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.NetworkPolicy llm/e2e-same-id-a-f3467868-prometheus-scraping 6m35s Normal SuccessfulCreate replicaset/e2e-same-id-a-f3467868-kserve-6ff68659d9 Created pod: e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.PodMonitor llm/e2e-same-id-a-f3467868-kserve-llmisvc-engine-default 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.DestinationRule llm/e2e-same-id-a-f3467868-kserve-workload-svc 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.HTTPRoute llm/e2e-same-id-a-f3467868-kserve-route 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.PodMonitor llm/e2e-same-id-a-f3467868-kserve-llmisvc-engine 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.Deployment llm/e2e-same-id-a-f3467868-kserve 6m35s Normal Created llminferenceservice/e2e-same-id-a-f3467868 Created v1.Secret llm/e2e-same-id-a-f3467868-kserve-self-signed-certs 6m34s Normal Started pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Container started 6m34s Normal AddedInterface pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Add eth0 [10.133.0.221/23] from ovn-kubernetes 6m34s Normal Pulled pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod 6m34s Normal Created pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Container created 6m31s Normal Updated llminferenceservice/e2e-same-id-a-f3467868 Updated v1.Secret llm/e2e-same-id-a-f3467868-kserve-self-signed-certs 6m23s Normal LLMInferenceServiceReady llminferenceservice/e2e-same-id-a-f3467868 LLMInferenceService [e2e-same-id-a-f3467868] is Ready 6m18s Normal ScalingReplicaSet deployment/e2e-same-id-b-f3467868-kserve Scaled up replica set e2e-same-id-b-f3467868-kserve-fbf6d69b5 from 0 to 1 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.Secret llm/e2e-same-id-b-f3467868-kserve-self-signed-certs 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.NetworkPolicy llm/e2e-same-id-b-f3467868-prometheus-scraping 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.PodMonitor llm/e2e-same-id-b-f3467868-kserve-llmisvc-engine 6m18s Normal Pulled pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod 6m18s Normal Created pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Container created 6m18s Normal Started pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Container started 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.PodMonitor llm/e2e-same-id-b-f3467868-kserve-llmisvc-engine-default 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.DestinationRule llm/e2e-same-id-b-f3467868-kserve-workload-svc 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.Deployment llm/e2e-same-id-b-f3467868-kserve 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.HTTPRoute llm/e2e-same-id-b-f3467868-kserve-route 6m18s Normal AddedInterface pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Add eth0 [10.133.0.222/23] from ovn-kubernetes 6m18s Normal SuccessfulCreate replicaset/e2e-same-id-b-f3467868-kserve-fbf6d69b5 Created pod: e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l 6m18s Normal Created llminferenceservice/e2e-same-id-b-f3467868 Created v1.Service llm/e2e-same-id-b-f3467868-kserve-workload-svc 6m14s Normal Updated llminferenceservice/e2e-same-id-b-f3467868 Updated v1.Secret llm/e2e-same-id-b-f3467868-kserve-self-signed-certs 6m7s Normal LLMInferenceServiceReady llminferenceservice/e2e-same-id-b-f3467868 LLMInferenceService [e2e-same-id-b-f3467868] is Ready 6m Warning ModelNameCollision llminferenceservice/e2e-same-id-b-f3467868 model name "test/e2e-same-model-id" overlaps with e2e-same-id-a-f3467868 in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-b-f3467868; safe to ignore if traffic splitting is planned 6m Warning ModelNameCollision llminferenceservice/e2e-same-id-a-f3467868 model name "test/e2e-same-model-id" overlaps with e2e-same-id-b-f3467868 in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-a-f3467868; safe to ignore if traffic splitting is planned 5m51s Normal Killing pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Stopping container main 5m50s Normal Killing pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Stopping container main 5m35s Warning Unhealthy pod/e2e-same-id-a-f3467868-kserve-6ff68659d9-6x26m Readiness probe failed: Get "https://10.133.0.221:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers) 5m33s Warning Unhealthy pod/e2e-same-id-b-f3467868-kserve-fbf6d69b5-n765l Readiness probe failed: Get "https://10.133.0.222:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)