| Time | Namespace | Component | RelatedObject | Reason | Message |
|---|---|---|---|---|---|
kserve-ci-e2e-test |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g to ip-10-0-128-42.ec2.internal | ||
kuadrant-system |
limitador-operator-controller-manager-6f9f468797-d7wdw |
Scheduled |
Successfully assigned kuadrant-system/limitador-operator-controller-manager-6f9f468797-d7wdw to ip-10-0-133-160.ec2.internal | ||
kuadrant-system |
limitador-limitador-ff68ff5cb-g8j52 |
Scheduled |
Successfully assigned kuadrant-system/limitador-limitador-ff68ff5cb-g8j52 to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z to ip-10-0-132-130.ec2.internal | ||
kuadrant-system |
limitador-limitador-69574b596d-nzkh5 |
Scheduled |
Successfully assigned kuadrant-system/limitador-limitador-69574b596d-nzkh5 to ip-10-0-133-160.ec2.internal | ||
kserve-ci-e2e-test |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
Scheduled |
Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm to ip-10-0-128-42.ec2.internal | ||
kuadrant-system |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd |
Scheduled |
Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx to ip-10-0-128-42.ec2.internal | ||
kuadrant-system |
kuadrant-console-plugin-85fd4b8b9d-m6wrp |
Scheduled |
Successfully assigned kuadrant-system/kuadrant-console-plugin-85fd4b8b9d-m6wrp to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf to ip-10-0-128-42.ec2.internal | ||
kuadrant-system |
authorino-operator-6d85f6564-c4d5z |
Scheduled |
Successfully assigned kuadrant-system/authorino-operator-6d85f6564-c4d5z to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
router-gateway-1-openshift-default-54c47cdfc7-kfbtn |
Scheduled |
Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-kfbtn to ip-10-0-133-160.ec2.internal | ||
kuadrant-system |
authorino-b6c89bb85-xnpk7 |
Scheduled |
Successfully assigned kuadrant-system/authorino-b6c89bb85-xnpk7 to ip-10-0-128-42.ec2.internal | ||
openshift-ingress |
router-default-568468f9f4-9h48n |
Scheduled |
Successfully assigned openshift-ingress/router-default-568468f9f4-9h48n to ip-10-0-133-160.ec2.internal | ||
kserve-ci-e2e-test |
fc-concurrency-test-kserve-b5d44467c-xdjqc |
Scheduled |
Successfully assigned kserve-ci-e2e-test/fc-concurrency-test-kserve-b5d44467c-xdjqc to ip-10-0-128-42.ec2.internal | ||
kuadrant-system |
authorino-8df697c9f-vs8lj |
Scheduled |
Successfully assigned kuadrant-system/authorino-8df697c9f-vs8lj to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs to ip-10-0-133-160.ec2.internal | ||
kserve-ci-e2e-test |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk to ip-10-0-128-42.ec2.internal | ||
kuadrant-system |
authorino-7c77957575-xc96s |
Scheduled |
Successfully assigned kuadrant-system/authorino-7c77957575-xc96s to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
router-gateway-1-openshift-default-54c47cdfc7-qbmpr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-54c47cdfc7-qbmpr to ip-10-0-133-160.ec2.internal | ||
kserve-ci-e2e-test |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Scheduled |
Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-799f46c59b-gx98l to ip-10-0-133-160.ec2.internal | ||
kserve-ci-e2e-test |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
Scheduled |
Successfully assigned kserve-ci-e2e-test/autoscale-hpa-deploy-kserve-66746bd8b9-f297r to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Scheduled |
Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
Scheduled |
Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 to ip-10-0-132-130.ec2.internal | ||
openshift-ingress |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Scheduled |
Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-769cdd8d88-mfr9j to ip-10-0-133-160.ec2.internal | ||
kserve-ci-e2e-test |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk |
Scheduled |
Successfully assigned kserve-ci-e2e-test/fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Scheduled |
Successfully assigned kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Scheduled |
Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
Scheduled |
Successfully assigned kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr to ip-10-0-128-42.ec2.internal | ||
kserve-ci-e2e-test |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Scheduled |
Successfully assigned kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l to ip-10-0-132-130.ec2.internal | ||
kserve-ci-e2e-test |
fc-smoke-test-kserve-7f4687897-r5tjx |
Scheduled |
Successfully assigned kserve-ci-e2e-test/fc-smoke-test-kserve-7f4687897-r5tjx to ip-10-0-133-160.ec2.internal | ||
kuadrant-system |
dns-operator-controller-manager-65b49595d7-m7vkn |
Scheduled |
Successfully assigned kuadrant-system/dns-operator-controller-manager-65b49595d7-m7vkn to ip-10-0-132-130.ec2.internal | ||
openshift-ingress |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
Scheduled |
Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qnh6j to ip-10-0-132-130.ec2.internal | ||
openshift-ingress |
deployment-controller |
router-default |
ScalingReplicaSet |
Scaled up replica set router-default-568468f9f4 from 0 to 1 | |
| (x9) | openshift-ingress |
replicaset-controller |
router-default-568468f9f4 |
FailedCreate |
Error creating: pods "router-default-568468f9f4-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes |
openshift-ingress |
service-controller |
router-default |
UnAvailableLoadBalancer |
There are no available nodes for LoadBalancer | |
openshift-ingress |
service-controller |
router-default |
EnsuringLoadBalancer |
Ensuring load balancer | |
openshift-ingress |
service-controller |
router-default |
EnsuredLoadBalancer |
Ensured load balancer | |
| (x16) | openshift-ingress |
replicaset-controller |
router-default-568468f9f4 |
FailedCreate |
Error creating: pods "router-default-568468f9f4-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes |
| (x3) | openshift-ingress |
service-controller |
router-default |
UpdatedLoadBalancer |
Updated load balancer with new hosts |
openshift-ingress |
replicaset-controller |
router-default-568468f9f4 |
SuccessfulCreate |
Created pod: router-default-568468f9f4-9h48n | |
| (x6) | openshift-ingress |
kubelet |
router-default-568468f9f4-9h48n |
FailedMount |
MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found |
| (x6) | openshift-ingress |
kubelet |
router-default-568468f9f4-9h48n |
FailedMount |
MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt |
openshift-ingress |
kubelet |
router-default-568468f9f4-9h48n |
Started |
Started container router | |
openshift-ingress |
kubelet |
router-default-568468f9f4-9h48n |
Created |
Created container: router | |
openshift-ingress |
kubelet |
router-default-568468f9f4-9h48n |
Pulled |
Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0595b1d6de8e08ddb61ce581075357331193db6206d341c15cf213514ef0f693" already present on machine | |
openshift-ingress |
multus |
router-default-568468f9f4-9h48n |
AddedInterface |
Add eth0 [10.134.0.9/23] from ovn-kubernetes | |
openshift-ingress |
replicaset-controller |
istiod-openshift-gateway-94bb8fbfd |
SuccessfulCreate |
Created pod: istiod-openshift-gateway-94bb8fbfd-qnh6j | |
openshift-ingress |
deployment-controller |
istiod-openshift-gateway |
ScalingReplicaSet |
Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1 | |
openshift-ingress |
multus |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
AddedInterface |
Add eth0 [10.133.0.37/23] from ovn-kubernetes | |
openshift-ingress |
kubelet |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
Pulling |
Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" | |
openshift-ingress |
kubelet |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
Started |
Started container discovery | |
openshift-ingress |
kubelet |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
Created |
Created container: discovery | |
openshift-ingress |
kubelet |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
Pulled |
Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 3.224s (3.224s including waiting). Image size: 181034515 bytes. | |
openshift-ingress |
kubelet |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 | |
openshift-ingress |
kubelet |
istiod-openshift-gateway-94bb8fbfd-qnh6j |
ProbeError |
Readiness probe error: HTTP probe failed with statuscode: 503 body: | |
openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Pulling |
Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" | |
openshift-ingress |
service-controller |
openshift-ai-inference-openshift-default |
EnsuringLoadBalancer |
Ensuring load balancer | |
openshift-ingress |
gateway_labeler_controller |
openshift-ai-inference |
AddedLabel |
Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference | |
openshift-ingress |
replicaset-controller |
openshift-ai-inference-openshift-default-769cdd8d88 |
SuccessfulCreate |
Created pod: openshift-ai-inference-openshift-default-769cdd8d88-mfr9j | |
openshift-ingress |
multus |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
AddedInterface |
Add eth0 [10.134.0.25/23] from ovn-kubernetes | |
openshift-ingress |
deployment-controller |
openshift-ai-inference-openshift-default |
ScalingReplicaSet |
Scaled up replica set openshift-ai-inference-openshift-default-769cdd8d88 from 0 to 1 | |
openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Pulled |
Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.754s (2.754s including waiting). Image size: 181251556 bytes. | |
openshift-ingress |
service-controller |
openshift-ai-inference-openshift-default |
EnsuredLoadBalancer |
Ensured load balancer | |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
RequirementsNotMet |
one or more requirements couldn't be found | |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
RequirementsUnknown |
requirements not yet checked | |
| (x2) | openshift-ingress |
horizontal-pod-autoscaler |
istiod-openshift-gateway |
FailedComputeMetricsReplicas |
invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
| (x2) | openshift-ingress |
horizontal-pod-autoscaler |
istiod-openshift-gateway |
FailedGetResourceMetric |
failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
AllRequirementsMet |
all requirements found, attempting install | |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
InstallSucceeded |
waiting for install components to report healthy | |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
InstallWaiting |
installing: waiting for deployment dns-operator-controller-manager to become ready: waiting for spec update of deployment "dns-operator-controller-manager" to be observed... | |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
InstallWaiting |
installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability. | |
kuadrant-system |
replicaset-controller |
dns-operator-controller-manager-65b49595d7 |
SuccessfulCreate |
Created pod: dns-operator-controller-manager-65b49595d7-m7vkn | |
kuadrant-system |
deployment-controller |
dns-operator-controller-manager |
ScalingReplicaSet |
Scaled up replica set dns-operator-controller-manager-65b49595d7 from 0 to 1 | |
kuadrant-system |
operator-lifecycle-manager |
rhcl-operator.v1.4.1 |
RequirementsUnknown |
requirements not yet checked | |
kuadrant-system |
operator-lifecycle-manager |
rhcl-operator.v1.4.1 |
RequirementsNotMet |
one or more requirements couldn't be found | |
kuadrant-system |
multus |
dns-operator-controller-manager-65b49595d7-m7vkn |
AddedInterface |
Add eth0 [10.133.0.42/23] from ovn-kubernetes | |
kuadrant-system |
kubelet |
dns-operator-controller-manager-65b49595d7-m7vkn |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5" | |
kuadrant-system |
dns-operator-controller-manager-65b49595d7-m7vkn_4fc48d8a-9baa-48b5-86ba-789ebdaa3766 |
a3f98d6c.kuadrant.io |
LeaderElection |
dns-operator-controller-manager-65b49595d7-m7vkn_4fc48d8a-9baa-48b5-86ba-789ebdaa3766 became leader | |
kuadrant-system |
kubelet |
dns-operator-controller-manager-65b49595d7-m7vkn |
Created |
Created container: manager | |
kuadrant-system |
kubelet |
dns-operator-controller-manager-65b49595d7-m7vkn |
Started |
Started container manager | |
kuadrant-system |
kubelet |
dns-operator-controller-manager-65b49595d7-m7vkn |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:ba46f54446ecd3ed1a3d877dcbb7359ff8f2d918827f3411a5cb0781915a65d5" in 2.818s (2.818s including waiting). Image size: 190472853 bytes. | |
kuadrant-system |
replicaset-controller |
kuadrant-operator-controller-manager-bb7ddb7d7 |
SuccessfulCreate |
Created pod: kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd | |
kuadrant-system |
operator-lifecycle-manager |
rhcl-operator.v1.4.1 |
InstallSucceeded |
waiting for install components to report healthy | |
kuadrant-system |
operator-lifecycle-manager |
rhcl-operator.v1.4.1 |
InstallWaiting |
installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability. | |
kuadrant-system |
operator-lifecycle-manager |
rhcl-operator.v1.4.1 |
AllRequirementsMet |
all requirements found, attempting install | |
kuadrant-system |
deployment-controller |
kuadrant-operator-controller-manager |
ScalingReplicaSet |
Scaled up replica set kuadrant-operator-controller-manager-bb7ddb7d7 from 0 to 1 | |
kuadrant-system |
operator-lifecycle-manager |
authorino-operator.v1.4.1 |
RequirementsNotMet |
one or more requirements couldn't be found | |
kuadrant-system |
operator-lifecycle-manager |
authorino-operator.v1.4.1 |
RequirementsUnknown |
requirements not yet checked | |
kuadrant-system |
kubelet |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3" | |
kuadrant-system |
multus |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd |
AddedInterface |
Add eth0 [10.133.0.43/23] from ovn-kubernetes | |
kuadrant-system |
operator-lifecycle-manager |
authorino-operator.v1.4.1 |
InstallSucceeded |
waiting for install components to report healthy | |
kuadrant-system |
operator-lifecycle-manager |
authorino-operator.v1.4.1 |
InstallWaiting |
installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability. | |
kuadrant-system |
replicaset-controller |
authorino-operator-6d85f6564 |
SuccessfulCreate |
Created pod: authorino-operator-6d85f6564-c4d5z | |
kuadrant-system |
deployment-controller |
authorino-operator |
ScalingReplicaSet |
Scaled up replica set authorino-operator-6d85f6564 from 0 to 1 | |
kuadrant-system |
operator-lifecycle-manager |
limitador-operator.v1.4.0 |
RequirementsUnknown |
requirements not yet checked | |
kuadrant-system |
operator-lifecycle-manager |
authorino-operator.v1.4.1 |
AllRequirementsMet |
all requirements found, attempting install | |
kuadrant-system |
operator-lifecycle-manager |
limitador-operator.v1.4.0 |
RequirementsNotMet |
one or more requirements couldn't be found | |
kuadrant-system |
multus |
authorino-operator-6d85f6564-c4d5z |
AddedInterface |
Add eth0 [10.133.0.44/23] from ovn-kubernetes | |
kuadrant-system |
kubelet |
authorino-operator-6d85f6564-c4d5z |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687" | |
kuadrant-system |
kubelet |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:0b77e4a874b620095a0300877aea53e78559a601ce62b194537ed21c2743bbf3" in 4.238s (4.239s including waiting). Image size: 425661757 bytes. | |
kuadrant-system |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_3278e50e-84eb-47cc-94bc-f3f397c6b61c |
f139389e.kuadrant.io |
LeaderElection |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd_3278e50e-84eb-47cc-94bc-f3f397c6b61c became leader | |
kuadrant-system |
kubelet |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd |
Created |
Created container: manager | |
kuadrant-system |
kubelet |
kuadrant-operator-controller-manager-bb7ddb7d7-fp8pd |
Started |
Started container manager | |
kuadrant-system |
multus |
limitador-operator-controller-manager-6f9f468797-d7wdw |
AddedInterface |
Add eth0 [10.134.0.26/23] from ovn-kubernetes | |
kuadrant-system |
operator-lifecycle-manager |
limitador-operator.v1.4.0 |
InstallSucceeded |
waiting for install components to report healthy | |
kuadrant-system |
operator-lifecycle-manager |
limitador-operator.v1.4.0 |
AllRequirementsMet |
all requirements found, attempting install | |
kuadrant-system |
replicaset-controller |
limitador-operator-controller-manager-6f9f468797 |
SuccessfulCreate |
Created pod: limitador-operator-controller-manager-6f9f468797-d7wdw | |
kuadrant-system |
deployment-controller |
limitador-operator-controller-manager |
ScalingReplicaSet |
Scaled up replica set limitador-operator-controller-manager-6f9f468797 from 0 to 1 | |
kuadrant-system |
kubelet |
limitador-operator-controller-manager-6f9f468797-d7wdw |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e" | |
kuadrant-system |
operator-lifecycle-manager |
limitador-operator.v1.4.0 |
InstallWaiting |
installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability. | |
kuadrant-system |
kubelet |
kuadrant-console-plugin-85fd4b8b9d-m6wrp |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a" | |
kuadrant-system |
replicaset-controller |
kuadrant-console-plugin-85fd4b8b9d |
SuccessfulCreate |
Created pod: kuadrant-console-plugin-85fd4b8b9d-m6wrp | |
kuadrant-system |
deployment-controller |
kuadrant-console-plugin |
ScalingReplicaSet |
Scaled up replica set kuadrant-console-plugin-85fd4b8b9d from 0 to 1 | |
kuadrant-system |
multus |
kuadrant-console-plugin-85fd4b8b9d-m6wrp |
AddedInterface |
Add eth0 [10.132.0.19/23] from ovn-kubernetes | |
kuadrant-system |
operator-lifecycle-manager |
dns-operator.v1.4.0 |
InstallSucceeded |
install strategy completed with no errors | |
kuadrant-system |
kubelet |
limitador-operator-controller-manager-6f9f468797-d7wdw |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:183bad2a6f5872aea38f54ad507f7e005d6facb02b18769e0407c82799f96a3e" in 3.168s (3.168s including waiting). Image size: 178538972 bytes. | |
kuadrant-system |
kubelet |
limitador-operator-controller-manager-6f9f468797-d7wdw |
Created |
Created container: manager | |
kuadrant-system |
limitador-operator-controller-manager-6f9f468797-d7wdw_076b85de-8e30-4423-a772-6bb05c12f2f9 |
3745a16e.kuadrant.io |
LeaderElection |
limitador-operator-controller-manager-6f9f468797-d7wdw_076b85de-8e30-4423-a772-6bb05c12f2f9 became leader | |
kuadrant-system |
kubelet |
limitador-operator-controller-manager-6f9f468797-d7wdw |
Started |
Started container manager | |
kuadrant-system |
kubelet |
kuadrant-console-plugin-85fd4b8b9d-m6wrp |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:6e69f41d3d70b0afa7691a5d2400065d3bcdec2ea82830cf2992b8e69ba06e2a" in 5.357s (5.357s including waiting). Image size: 342502139 bytes. | |
kuadrant-system |
kubelet |
kuadrant-console-plugin-85fd4b8b9d-m6wrp |
Started |
Started container kuadrant-console-plugin | |
kuadrant-system |
kubelet |
kuadrant-console-plugin-85fd4b8b9d-m6wrp |
Created |
Created container: kuadrant-console-plugin | |
kuadrant-system |
authorino-operator-6d85f6564-c4d5z_11cf079c-f6d8-4cfd-8733-4797902dd98a |
aac3a15d.authorino.kuadrant.io |
LeaderElection |
authorino-operator-6d85f6564-c4d5z_11cf079c-f6d8-4cfd-8733-4797902dd98a became leader | |
kuadrant-system |
kubelet |
authorino-operator-6d85f6564-c4d5z |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:b2cc2c0f0729dad98f970830816e300c0b961ba42491f362295512d886384687" in 9.781s (9.781s including waiting). Image size: 168639631 bytes. | |
kuadrant-system |
kubelet |
authorino-operator-6d85f6564-c4d5z |
Created |
Created container: manager | |
kuadrant-system |
kubelet |
authorino-operator-6d85f6564-c4d5z |
Started |
Started container manager | |
kuadrant-system |
operator-lifecycle-manager |
rhcl-operator.v1.4.1 |
InstallSucceeded |
install strategy completed with no errors | |
kuadrant-system |
operator-lifecycle-manager |
limitador-operator.v1.4.0 |
InstallSucceeded |
install strategy completed with no errors | |
kuadrant-system |
operator-lifecycle-manager |
authorino-operator.v1.4.1 |
InstallSucceeded |
install strategy completed with no errors | |
kuadrant-system |
deployment-controller |
limitador-limitador |
ScalingReplicaSet |
Scaled up replica set limitador-limitador-ff68ff5cb from 0 to 1 | |
kuadrant-system |
replicaset-controller |
limitador-limitador-69574b596d |
SuccessfulCreate |
Created pod: limitador-limitador-69574b596d-nzkh5 | |
kuadrant-system |
multus |
limitador-limitador-ff68ff5cb-g8j52 |
AddedInterface |
Add eth0 [10.133.0.45/23] from ovn-kubernetes | |
kuadrant-system |
replicaset-controller |
limitador-limitador-ff68ff5cb |
SuccessfulCreate |
Created pod: limitador-limitador-ff68ff5cb-g8j52 | |
kuadrant-system |
deployment-controller |
limitador-limitador |
ScalingReplicaSet |
Scaled up replica set limitador-limitador-69574b596d from 0 to 1 | |
kuadrant-system |
replicaset-controller |
authorino-b6c89bb85 |
SuccessfulCreate |
Created pod: authorino-b6c89bb85-xnpk7 | |
kuadrant-system |
deployment-controller |
authorino |
ScalingReplicaSet |
Scaled up replica set authorino-b6c89bb85 from 0 to 1 | |
kuadrant-system |
kubelet |
limitador-limitador-ff68ff5cb-g8j52 |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" | |
kuadrant-system |
multus |
limitador-limitador-69574b596d-nzkh5 |
AddedInterface |
Add eth0 [10.134.0.27/23] from ovn-kubernetes | |
kuadrant-system |
kubelet |
limitador-limitador-69574b596d-nzkh5 |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" | |
kuadrant-system |
multus |
authorino-b6c89bb85-xnpk7 |
AddedInterface |
Add eth0 [10.132.0.20/23] from ovn-kubernetes | |
kuadrant-system |
multus |
authorino-8df697c9f-vs8lj |
AddedInterface |
Add eth0 [10.132.0.21/23] from ovn-kubernetes | |
kuadrant-system |
deployment-controller |
authorino |
ScalingReplicaSet |
Scaled up replica set authorino-8df697c9f from 0 to 1 | |
kuadrant-system |
kubelet |
authorino-8df697c9f-vs8lj |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" | |
kuadrant-system |
replicaset-controller |
authorino-8df697c9f |
SuccessfulCreate |
Created pod: authorino-8df697c9f-vs8lj | |
kuadrant-system |
kubelet |
authorino-b6c89bb85-xnpk7 |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" | |
kuadrant-system |
kubelet |
authorino-b6c89bb85-xnpk7 |
Created |
Created container: authorino | |
kuadrant-system |
kubelet |
authorino-b6c89bb85-xnpk7 |
Started |
Started container authorino | |
kuadrant-system |
kubelet |
authorino-8df697c9f-vs8lj |
Started |
Started container authorino | |
kuadrant-system |
kubelet |
authorino-8df697c9f-vs8lj |
Created |
Created container: authorino | |
kuadrant-system |
kubelet |
authorino-b6c89bb85-xnpk7 |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 2.124s (2.124s including waiting). Image size: 202706732 bytes. | |
kuadrant-system |
kubelet |
authorino-8df697c9f-vs8lj |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 1.851s (1.851s including waiting). Image size: 202706732 bytes. | |
kuadrant-system |
kubelet |
limitador-limitador-ff68ff5cb-g8j52 |
Created |
Created container: limitador | |
kuadrant-system |
replicaset-controller |
authorino-b6c89bb85 |
SuccessfulDelete |
Deleted pod: authorino-b6c89bb85-xnpk7 | |
kuadrant-system |
kubelet |
limitador-limitador-69574b596d-nzkh5 |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 4.031s (4.031s including waiting). Image size: 250499765 bytes. | |
kuadrant-system |
kubelet |
limitador-limitador-ff68ff5cb-g8j52 |
Started |
Started container limitador | |
kuadrant-system |
kubelet |
limitador-limitador-69574b596d-nzkh5 |
Created |
Created container: limitador | |
kuadrant-system |
kubelet |
limitador-limitador-69574b596d-nzkh5 |
Started |
Started container limitador | |
kuadrant-system |
kubelet |
limitador-limitador-ff68ff5cb-g8j52 |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:0b02c000d9a175235223ec5916345a1ccedeedc25c87cc0638e8554e6c318474" in 3.785s (3.785s including waiting). Image size: 250499765 bytes. | |
kuadrant-system |
deployment-controller |
authorino |
ScalingReplicaSet |
Scaled down replica set authorino-b6c89bb85 from 1 to 0 | |
kuadrant-system |
kubelet |
authorino-b6c89bb85-xnpk7 |
Killing |
Stopping container authorino | |
kuadrant-system |
deployment-controller |
limitador-limitador |
ScalingReplicaSet |
Scaled down replica set limitador-limitador-ff68ff5cb from 1 to 0 | |
kuadrant-system |
replicaset-controller |
limitador-limitador-ff68ff5cb |
SuccessfulDelete |
Deleted pod: limitador-limitador-ff68ff5cb-g8j52 | |
kuadrant-system |
kubelet |
limitador-limitador-ff68ff5cb-g8j52 |
Killing |
Stopping container limitador | |
kuadrant-system |
kubelet |
authorino-7c77957575-xc96s |
Pulling |
Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" | |
kuadrant-system |
replicaset-controller |
authorino-7c77957575 |
SuccessfulCreate |
Created pod: authorino-7c77957575-xc96s | |
kuadrant-system |
multus |
authorino-7c77957575-xc96s |
AddedInterface |
Add eth0 [10.132.0.22/23] from ovn-kubernetes | |
kuadrant-system |
deployment-controller |
authorino |
ScalingReplicaSet |
Scaled up replica set authorino-7c77957575 from 0 to 1 | |
kuadrant-system |
kubelet |
authorino-7c77957575-xc96s |
Pulled |
Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:eaadd50c0e4938926e50283a5f3e3b5f1dd24d820a3a2c7434153080180cc95a" in 581ms (581ms including waiting). Image size: 202706732 bytes. | |
kuadrant-system |
kubelet |
authorino-7c77957575-xc96s |
Started |
Started container authorino | |
kuadrant-system |
kubelet |
authorino-7c77957575-xc96s |
Created |
Created container: authorino | |
kuadrant-system |
kubelet |
authorino-8df697c9f-vs8lj |
Killing |
Stopping container authorino | |
kuadrant-system |
deployment-controller |
authorino |
ScalingReplicaSet |
Scaled down replica set authorino-8df697c9f from 1 to 0 | |
kuadrant-system |
replicaset-controller |
authorino-8df697c9f |
SuccessfulDelete |
Deleted pod: authorino-8df697c9f-vs8lj | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve | |
kserve-ci-e2e-test |
deployment-controller |
scheduler-v06-pd-migration-test-kserve-prefill |
ScalingReplicaSet |
Scaled up replica set scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4 from 0 to 1 | |
kserve-ci-e2e-test |
deployment-controller |
scheduler-v06-pd-migration-test-kserve |
ScalingReplicaSet |
Scaled up replica set scheduler-v06-pd-migration-test-kserve-68f9854ff from 0 to 1 | |
| (x12) | kserve-ci-e2e-test |
OpenDataHubModelController |
scheduler-v06-pd-migration-test |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-pd-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-pd-migration-test-kserve-route" not found |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb | |
kserve-ci-e2e-test |
deployment-controller |
scheduler-v06-pd-migration-test-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set scheduler-v06-pd-migration-test-kserve-router-scheduler-79c5847dfd from 0 to 1 | |
kserve-ci-e2e-test |
multus |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
AddedInterface |
Add eth0 [10.133.0.49/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.Role kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-role | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-rb | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Pulling |
Pulling image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-prefill | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.Service kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-workload-svc | |
kserve-ci-e2e-test |
multus |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
AddedInterface |
Add eth0 [10.132.0.24/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-pd-migration-test-epp-sa | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Pulling |
Pulling image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" | |
kserve-ci-e2e-test |
replicaset-controller |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79c5847dfd |
SuccessfulCreate |
Created pod: scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l | |
kserve-ci-e2e-test |
replicaset-controller |
scheduler-v06-pd-migration-test-kserve-68f9854ff |
SuccessfulCreate |
Created pod: scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx | |
kserve-ci-e2e-test |
replicaset-controller |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4 |
SuccessfulCreate |
Created pod: scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q | |
kserve-ci-e2e-test |
multus |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
AddedInterface |
Add eth0 [10.133.0.50/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Pulling |
Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Pulled |
Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" in 1.754s (1.754s including waiting). Image size: 79178094 bytes. | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" in 2.55s (2.55s including waiting). Image size: 179512283 bytes. | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Started |
Started container llm-d-routing-sidecar | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Created |
Created container: llm-d-routing-sidecar | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" in 2.361s (2.361s including waiting). Image size: 166803393 bytes. | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Pulling |
Pulling image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Pulled |
Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" in 1.483s (1.483s including waiting). Image size: 79178094 bytes. | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Started |
Started container main | |
openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
ProbeError |
Readiness probe error: Get "http://10.134.0.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body: | |
openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Unhealthy |
Readiness probe failed: Get "http://10.134.0.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) | |
| (x2) | openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Created |
Created container: istio-proxy |
| (x2) | openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Started |
Started container istio-proxy |
openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Pulled |
Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/fc-smoke-test-epp-rb | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.Service kserve-ci-e2e-test/fc-smoke-test-kserve-workload-svc | |
kserve-ci-e2e-test |
multus |
fc-smoke-test-kserve-7f4687897-r5tjx |
AddedInterface |
Add eth0 [10.134.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-7f4687897-r5tjx |
Pulling |
Pulling image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" | |
kserve-ci-e2e-test |
replicaset-controller |
fc-smoke-test-kserve-7f4687897 |
SuccessfulCreate |
Created pod: fc-smoke-test-kserve-7f4687897-r5tjx | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.Role kserve-ci-e2e-test/fc-smoke-test-epp-role | |
kserve-ci-e2e-test |
replicaset-controller |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf |
SuccessfulCreate |
Created pod: fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.Service kserve-ci-e2e-test/fc-smoke-test-epp-service | |
kserve-ci-e2e-test |
deployment-controller |
fc-smoke-test-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set fc-smoke-test-kserve-router-scheduler-fcbfbd8bf from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/fc-smoke-test-kserve-router-scheduler | |
kserve-ci-e2e-test |
deployment-controller |
fc-smoke-test-kserve |
ScalingReplicaSet |
Scaled up replica set fc-smoke-test-kserve-7f4687897 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.Secret kserve-ci-e2e-test/fc-smoke-test-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/fc-smoke-test-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test-fc-smoke-test-epp-auth-rb | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/fc-smoke-test-epp-sa | |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk |
Pulling |
Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" | |
kserve-ci-e2e-test |
multus |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk |
AddedInterface |
Add eth0 [10.132.0.25/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-7f4687897-r5tjx |
Created |
Created container: main | |
| (x5) | openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
ProbeError |
Startup probe error: Get "http://10.134.0.25:15021/healthz/ready": dial tcp 10.134.0.25:15021: connect: connection refused body: |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk |
Pulled |
Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" in 1.538s (1.538s including waiting). Image size: 179512283 bytes. | |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-7f4687897-r5tjx |
Pulled |
Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" in 1.6s (1.6s including waiting). Image size: 79178094 bytes. | |
| (x5) | openshift-ingress |
kubelet |
openshift-ai-inference-openshift-default-769cdd8d88-mfr9j |
Unhealthy |
Startup probe failed: Get "http://10.134.0.25:15021/healthz/ready": dial tcp 10.134.0.25:15021: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-7f4687897-r5tjx |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
fc-smoke-test-kserve-router-scheduler-fcbfbd8bf-k8qnk |
Started |
Started container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Updated |
Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-self-signed-certs | |
| (x10) | kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-pd-migration-test-kserve-route | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/fc-smoke-test-kserve-route | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Updated |
Updated v1.Secret kserve-ci-e2e-test/fc-smoke-test-kserve-self-signed-certs | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/fc-smoke-test-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
LLMInferenceServiceReady |
LLMInferenceService [scheduler-v06-pd-migration-test] is Ready | |
| (x3) | kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/scheduler-v06-pd-migration-test; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
LLMInferenceServiceReady |
LLMInferenceService [fc-smoke-test] is Ready | |
| (x5) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with scheduler-v06-pd-migration-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-68f9854ff-82kzx |
Killing |
Stopping container llm-d-routing-sidecar | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-prefill-794df6ffb4-4cx7q |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.49:8000/health": dial tcp 10.133.0.49:8000: connect: connection refused | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-pd-migration-test |
Deleted |
Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-v06-pd-migration-test-epp-auth-rb | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Killing |
Stopping container main | |
| (x2) | kserve-ci-e2e-test |
kubelet |
scheduler-v06-pd-migration-test-kserve-router-scheduler-79ww96l |
Unhealthy |
Readiness probe failed: service unhealthy (responded with "NOT_SERVING") |
| (x10) | kserve-ci-e2e-test |
OpenDataHubModelController |
scheduler-v06-threshold-migration-test |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-v06-threshold-migration-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-v06-threshold-migration-test-kserve-route" not found |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.Role kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-role | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.Service kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-workload-svc | |
kserve-ci-e2e-test |
replicaset-controller |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57 |
SuccessfulCreate |
Created pod: scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-epp-sa | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0 | |
kserve-ci-e2e-test |
replicaset-controller |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fbd55d |
SuccessfulCreate |
Created pod: scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs | |
kserve-ci-e2e-test |
deployment-controller |
scheduler-v06-threshold-migration-test-kserve |
ScalingReplicaSet |
Scaled up replica set scheduler-v06-threshold-migration-test-kserve-5d54f65f57 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve | |
kserve-ci-e2e-test |
deployment-controller |
scheduler-v06-threshold-migration-test-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set scheduler-v06-threshold-migration-test-kserve-router-scheduler-5db9bd4595 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
scheduler-v06-threshold-migration-test-kserve-router-scheduler-5db9bd4595 |
SuccessfulCreate |
Created pod: scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-prefill | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve | |
kserve-ci-e2e-test |
deployment-controller |
scheduler-v06-threshold-migration-test-kserve-prefill |
ScalingReplicaSet |
Scaled up replica set scheduler-v06-threshold-migration-test-kserve-prefill-b58fbd55d from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-rb | |
kserve-ci-e2e-test |
multus |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
AddedInterface |
Add eth0 [10.132.0.26/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
Created v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Started |
Started container llm-d-routing-sidecar | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Created |
Created container: llm-d-routing-sidecar | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-disagg-sidecar:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
Created |
Created container: main | |
kserve-ci-e2e-test |
multus |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
AddedInterface |
Add eth0 [10.133.0.51/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
AddedInterface |
Add eth0 [10.133.0.52/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
FailedToRetrieveImagePullSecret |
Unable to retrieve some image pull secrets (scheduler-v06-threshold-migration-test-epp-sa-dockercfg-klsww); attempting to pull the image may not succeed. | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
Started |
Started container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with scheduler-v06-threshold-migration-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Updated |
Updated v1.Secret kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-route | |
| (x10) | kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-v06-threshold-migration-test-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
LLMInferenceServiceReady |
LLMInferenceService [scheduler-v06-threshold-migration-test] is Ready | |
| (x7) | kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/scheduler-v06-threshold-migration-test; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
scheduler-v06-threshold-migration-test |
Deleted |
Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test0a07e82be2036793543b2d3db2ea0785-scheduler-v0 | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Killing |
Stopping container llm-d-routing-sidecar | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-5d54f65f57-27ctz |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-prefill-b58fmzjxs |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
scheduler-v06-threshold-migration-test-kserve-router-sched6ws58 |
Unhealthy |
Readiness probe failed: service unhealthy (responded with "NOT_SERVING") | |
| (x10) | kserve-ci-e2e-test |
OpenDataHubModelController |
precise-prefix-cache-test |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found |
kserve-ci-e2e-test |
deployment-controller |
precise-prefix-cache-test-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-75779d4b5 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" | |
kserve-ci-e2e-test |
replicaset-controller |
precise-prefix-cache-test-kserve-6d6c8b5f7c |
SuccessfulCreate |
Created pod: precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz | |
kserve-ci-e2e-test |
replicaset-controller |
precise-prefix-cache-test-kserve-6d6c8b5f7c |
SuccessfulCreate |
Created pod: precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz | |
kserve-ci-e2e-test |
replicaset-controller |
precise-prefix-cache-test-kserve-router-scheduler-75779d4b5 |
SuccessfulCreate |
Created pod: precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa | |
kserve-ci-e2e-test |
multus |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
AddedInterface |
Add eth0 [10.133.0.53/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
AddedInterface |
Add eth0 [10.132.0.27/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" already present on machine | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role | |
kserve-ci-e2e-test |
deployment-controller |
precise-prefix-cache-test-kserve |
ScalingReplicaSet |
Scaled up replica set precise-prefix-cache-test-kserve-6d6c8b5f7c from 0 to 2 | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Pulling |
Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" | |
kserve-ci-e2e-test |
multus |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
AddedInterface |
Add eth0 [10.133.0.54/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Started |
Started container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with precise-prefix-cache-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" in 4.016s (4.016s including waiting). Image size: 300595129 bytes. | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Pulled |
Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" in 4.327s (4.327s including waiting). Image size: 300595129 bytes. | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Pulling |
Pulling image "quay.io/opendatahub/llm-d-kv-cache:v0.8.0" | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route | |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Updated |
Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Started |
Started container tokenizer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Created |
Created container: tokenizer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Pulled |
Successfully pulled image "quay.io/opendatahub/llm-d-kv-cache:v0.8.0" in 43.867s (43.867s including waiting). Image size: 2935159967 bytes. | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Unhealthy |
Startup probe failed: Get "http://10.133.0.54:8082/healthz": dial tcp 10.133.0.54:8082: connect: connection refused | |
| (x3) | kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine |
| (x2) | kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
BackOff |
Back-off restarting failed container main in pod precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf_kserve-ci-e2e-test(9ff96acc-75a8-48e2-9709-8abb012acf3f) |
| (x3) | kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Created |
Created container: main |
| (x3) | kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Started |
Started container main |
| (x2) | kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Unhealthy |
Liveness probe failed: timeout: failed to connect service "10.133.0.54:9003" within 1s: context deadline exceeded |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
LLMInferenceServiceReady |
LLMInferenceService [precise-prefix-cache-test] is Ready | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/precise-prefix-cache-test; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
LLMInferenceServiceController |
precise-prefix-cache-test |
Deleted |
Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-nmsvz |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-6d6c8b5f7c-kgjrz |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Killing |
Stopping container tokenizer | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Unhealthy |
Readiness probe failed: service unhealthy (responded with "NOT_SERVING") | |
| (x2) | kserve-ci-e2e-test |
kubelet |
precise-prefix-cache-test-kserve-router-scheduler-75779d4bffhzf |
Unhealthy |
Readiness probe failed: Get "http://10.133.0.54:8082/healthz": dial tcp 10.133.0.54:8082: connect: connection refused |
| (x11) | kserve-ci-e2e-test |
OpenDataHubModelController |
llmisvc-router-managed-test-llm-e45d1f79 |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-e45d1f79-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-e45d1f79-kserve-route" not found |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-router-managed-test-llm-e45d1f79-kserve |
ScalingReplicaSet |
Scaled up replica set llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-rb | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5657fd78d9 from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler-5657fd78d9 |
SuccessfulCreate |
Created pod: llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66 |
SuccessfulCreate |
Created pod: llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-service | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-sa | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-epp-role | |
kserve-ci-e2e-test |
multus |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
AddedInterface |
Add eth0 [10.133.0.55/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-scheduler | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
multus |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
AddedInterface |
Add eth0 [10.132.0.28/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
Created |
Created container: main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with llmisvc-router-managed-test-llm-e45d1f79 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Updated |
Updated v1.Secret kserve-ci-e2e-test/llmisv5c7e67b6c51568d1d6d13829a9337f2a-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-route | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
Deleted |
Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testef4d2875be14b30dc1561ed84d0d4bde-llmisvc-rout | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
LLMInferenceServiceReady |
LLMInferenceService [llmisvc-router-managed-test-llm-e45d1f79] is Ready | |
| (x8) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-e45d1f79 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-e45d1f79; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-e45d1f79-kserve-df8675f66-sskh8 |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc5c7e67b6c51568d1d6d13829a9337f2a-kserve-router-schexb8dr |
Unhealthy |
Readiness probe failed: service unhealthy (responded with "NOT_SERVING") | |
| (x11) | kserve-ci-e2e-test |
OpenDataHubModelController |
llmisvc-router-managed-test-llm-5b1e8f15 |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-5b1e8f15-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-5b1e8f15-kserve-route" not found |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-role | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test7f54e84970003a6e7372bdbcb574f7ed-llmisvc-rout | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-sa | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977 |
SuccessfulCreate |
Created pod: llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 | |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-router-managed-test-llm-5b1e8f15-kserve |
ScalingReplicaSet |
Scaled up replica set llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk |
AddedInterface |
Add eth0 [10.132.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
multus |
llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 |
AddedInterface |
Add eth0 [10.133.0.56/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 |
Started |
Started container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler | |
kserve-ci-e2e-test |
kubelet |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-service | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-5b1e8f15-kserve-df5b5c977-xdvj2 |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk |
Started |
Started container main | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-69b67488bb |
SuccessfulCreate |
Created pod: llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-sche7q2hk | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-epp-rb | |
kserve-ci-e2e-test |
deployment-controller |
llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set llmisvce55ae740357a3a31a27cdb8b66ffe20f-kserve-router-scheduler-69b67488bb from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Updated |
Updated v1.Secret kserve-ci-e2e-test/llmisve55ae740357a3a31a27cdb8b66ffe20f-kserve-self-signed-certs | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15-kserve-route | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
LLMInferenceServiceReady |
LLMInferenceService [llmisvc-router-managed-test-llm-5b1e8f15] is Ready | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/fc-concurrency-test-kserve-router-scheduler | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.Secret kserve-ci-e2e-test/fc-concurrency-test-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/fc-concurrency-test-epp-sa | |
kserve-ci-e2e-test |
replicaset-controller |
fc-concurrency-test-kserve-router-scheduler-78c47c8645 |
SuccessfulCreate |
Created pod: fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.Service kserve-ci-e2e-test/fc-concurrency-test-epp-service | |
kserve-ci-e2e-test |
kubelet |
fc-concurrency-test-kserve-b5d44467c-xdjqc |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
deployment-controller |
fc-concurrency-test-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set fc-concurrency-test-kserve-router-scheduler-78c47c8645 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/fc-concurrency-test-epp-rb | |
kserve-ci-e2e-test |
multus |
fc-concurrency-test-kserve-b5d44467c-xdjqc |
AddedInterface |
Add eth0 [10.132.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.Role kserve-ci-e2e-test/fc-concurrency-test-epp-role | |
kserve-ci-e2e-test |
deployment-controller |
fc-concurrency-test-kserve |
ScalingReplicaSet |
Scaled up replica set fc-concurrency-test-kserve-b5d44467c from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test-fc-concurrency-test-epp-auth-rb | |
kserve-ci-e2e-test |
replicaset-controller |
fc-concurrency-test-kserve-b5d44467c |
SuccessfulCreate |
Created pod: fc-concurrency-test-kserve-b5d44467c-xdjqc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.Deployment kserve-ci-e2e-test/fc-concurrency-test-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
Created v1.Service kserve-ci-e2e-test/fc-concurrency-test-kserve-workload-svc | |
kserve-ci-e2e-test |
kubelet |
fc-concurrency-test-kserve-b5d44467c-xdjqc |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
fc-concurrency-test-kserve-b5d44467c-xdjqc |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx |
Created |
Created container: main | |
kserve-ci-e2e-test |
multus |
fc-concurrency-test-kserve-router-scheduler-78c47c8645-m62qx |
AddedInterface |
Add eth0 [10.133.0.57/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/fc-concurrency-test-kserve-route | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Updated |
Updated v1.Secret kserve-ci-e2e-test/fc-concurrency-test-kserve-self-signed-certs | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/fc-concurrency-test-kserve-shadow-svc |
| (x8) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-concurrency-test; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
LLMInferenceServiceReady |
LLMInferenceService [fc-concurrency-test] is Ready | |
| (x13) | kserve-ci-e2e-test |
OpenDataHubModelController |
llmisvc-router-managed-test-llm-4b931143 |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-router-managed-test-llm-4b931143-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-router-managed-test-llm-4b931143-kserve-route" not found |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.Role kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-role | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-6f9bc95cfd |
SuccessfulCreate |
Created pod: llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf | |
kserve-ci-e2e-test |
deployment-controller |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler-6f9bc95cfd from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-sa | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test8ac8e3d2264ccb939eb021b0b835847c-llmisvc-rout | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-rb | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969 |
SuccessfulCreate |
Created pod: llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs | |
kserve-ci-e2e-test |
multus |
llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs |
AddedInterface |
Add eth0 [10.134.0.29/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-router-managed-test-llm-4b931143-kserve-85bf86796966gbs |
Started |
Started container main | |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-router-managed-test-llm-4b931143-kserve |
ScalingReplicaSet |
Scaled up replica set llmisvc-router-managed-test-llm-4b931143-kserve-85bf867969 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-scheduler | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-epp-service | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-workload-svc | |
kserve-ci-e2e-test |
kubelet |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
multus |
llmisvcca2d2d7d499abb359505529ebe02c136-kserve-router-schep28gf |
AddedInterface |
Add eth0 [10.133.0.58/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Updated |
Updated v1.Secret kserve-ci-e2e-test/llmisvca2d2d7d499abb359505529ebe02c136-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-route | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143-kserve-shadow-svc |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
LLMInferenceServiceReady |
LLMInferenceService [llmisvc-router-managed-test-llm-4b931143] is Ready | |
kserve-ci-e2e-test |
replicaset-controller |
router-gateway-1-openshift-default-799f46c59b |
SuccessfulCreate |
Created pod: router-gateway-1-openshift-default-799f46c59b-gx98l | |
kserve-ci-e2e-test |
deployment-controller |
router-gateway-1-openshift-default |
ScalingReplicaSet |
Scaled up replica set router-gateway-1-openshift-default-799f46c59b from 0 to 1 | |
kserve-ci-e2e-test |
service-controller |
router-gateway-1-openshift-default |
EnsuringLoadBalancer |
Ensuring load balancer | |
kserve-ci-e2e-test |
multus |
router-gateway-1-openshift-default-799f46c59b-gx98l |
AddedInterface |
Add eth0 [10.134.0.30/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
service-controller |
router-gateway-1-openshift-default |
EnsuredLoadBalancer |
Ensured load balancer | |
kserve-ci-e2e-test |
deployment-controller |
gw-section-name-router-with-gat-2f0a622e-kserve |
ScalingReplicaSet |
Scaled up replica set gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
Created |
Created v1.Secret kserve-ci-e2e-test/gw-sec0c69dceeb48768325d1a53a749e65786-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
Created |
Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve | |
kserve-ci-e2e-test |
replicaset-controller |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494 |
SuccessfulCreate |
Created pod: gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf | |
kserve-ci-e2e-test |
multus |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
AddedInterface |
Add eth0 [10.132.0.31/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
Created |
Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
Created |
Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
Created |
Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-2f0a622e-kserve-route | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
Started |
Started container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-2f0a622e |
UpdateFailed |
Failed to update status for LLMInferenceService "gw-section-name-router-with-gat-2f0a622e": Operation cannot be fulfilled on llminferenceservices.serving.kserve.io "gw-section-name-router-with-gat-2f0a622e": the object has been modified; please apply your changes to the latest version and try again | |
| (x3) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
router-with-gateway-section-nam-5d237677 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/router-with-gateway-section-nam-5d237677: referenced by 1 LLMInferenceService(s) |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
router-with-managed-route-gw-se-010f4010 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/router-with-managed-route-gw-se-010f4010: referenced by 1 LLMInferenceService(s) |
| (x2) | kserve-ci-e2e-test |
gateway_labeler_controller |
router-gateway-1 |
AddedLabel |
Added label istio.io/rev=openshift-gateway to gateway router-gateway-1 |
kserve-ci-e2e-test |
replicaset-controller |
router-gateway-1-openshift-default-54c47cdfc7 |
SuccessfulCreate |
Created pod: router-gateway-1-openshift-default-54c47cdfc7-qbmpr | |
kserve-ci-e2e-test |
replicaset-controller |
router-gateway-1-openshift-default-54c47cdfc7 |
SuccessfulDelete |
Deleted pod: router-gateway-1-openshift-default-54c47cdfc7-qbmpr | |
kserve-ci-e2e-test |
deployment-controller |
router-gateway-1-openshift-default |
ScalingReplicaSet |
Scaled down replica set router-gateway-1-openshift-default-54c47cdfc7 from 1 to 0 | |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
model-fb-opt-125m-gw-section-na-e3d39686 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/model-fb-opt-125m-gw-section-na-e3d39686: referenced by 1 LLMInferenceService(s) |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
workload-llmd-simulator-gw-sect-0fb35c52 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/workload-llmd-simulator-gw-sect-0fb35c52: referenced by 1 LLMInferenceService(s) |
kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Unhealthy |
Readiness probe failed: Get "http://10.134.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) | |
| (x2) | kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Started |
Started container istio-proxy |
| (x2) | kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Pulled |
Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine |
| (x2) | kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Created |
Created container: istio-proxy |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-2f0a622e-kserve-8dc8c8494-xj8kf |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.31:8000/health": context deadline exceeded | |
| (x5) | kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Unhealthy |
Startup probe failed: Get "http://10.134.0.30:15021/healthz/ready": dial tcp 10.134.0.30:15021: connect: connection refused |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-f1d92d0f |
Created |
Created v1.Secret kserve-ci-e2e-test/gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-f1d92d0f |
Created |
Created v1.HTTPRoute kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-route | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-f1d92d0f |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-f1d92d0f |
Created |
Created v1.Service kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-f1d92d0f |
Created |
Created v1.Deployment kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve | |
kserve-ci-e2e-test |
deployment-controller |
gw-section-name-router-with-gat-f1d92d0f-kserve |
ScalingReplicaSet |
Scaled up replica set gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbf from 0 to 1 | |
kserve-ci-e2e-test |
replicaset-controller |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbf |
SuccessfulCreate |
Created pod: gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
multus |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
AddedInterface |
Add eth0 [10.132.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
gw-section-name-router-with-gat-f1d92d0f |
Created |
Created v1.DestinationRule kserve-ci-e2e-test/gw-section-name-router-with-gat-f1d92d0f-kserve-workload-svc | |
| (x3) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
router-with-gateway-ref-gw-sect-bd0c4d94 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/router-with-gateway-ref-gw-sect-bd0c4d94: referenced by 1 LLMInferenceService(s) |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
model-fb-opt-125m-gw-section-na-8bf35153 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/model-fb-opt-125m-gw-section-na-8bf35153: referenced by 1 LLMInferenceService(s) |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
router-with-managed-route-gw-se-bc1f68e4 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/router-with-managed-route-gw-se-bc1f68e4: referenced by 1 LLMInferenceService(s) |
kserve-ci-e2e-test |
replicaset-controller |
router-gateway-1-openshift-default-54c47cdfc7 |
SuccessfulCreate |
Created pod: router-gateway-1-openshift-default-54c47cdfc7-kfbtn | |
| (x2) | kserve-ci-e2e-test |
deployment-controller |
router-gateway-1-openshift-default |
ScalingReplicaSet |
Scaled up replica set router-gateway-1-openshift-default-54c47cdfc7 from 0 to 1 |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceConfigController |
workload-llmd-simulator-gw-sect-81946990 |
DeletionBlocked |
Cannot delete LLMInferenceServiceConfig kserve-ci-e2e-test/workload-llmd-simulator-gw-sect-81946990: referenced by 1 LLMInferenceService(s) |
kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-54c47cdfc7-kfbtn |
Started |
Started container istio-proxy | |
kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-54c47cdfc7-kfbtn |
Pulled |
Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine | |
kserve-ci-e2e-test |
multus |
router-gateway-1-openshift-default-54c47cdfc7-kfbtn |
AddedInterface |
Add eth0 [10.134.0.32/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-54c47cdfc7-kfbtn |
Created |
Created container: istio-proxy | |
| (x6) | kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-54c47cdfc7-kfbtn |
Unhealthy |
Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
Killing |
Stopping container main | |
| (x2) | kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
FailedMount |
MountVolume.SetUp failed for volume "tls-certs" : secret "gw-sec2774c263d49959f50d9eebc552e13bf9-kserve-self-signed-certs" not found |
kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Killing |
Stopping container istio-proxy | |
kserve-ci-e2e-test |
deployment-controller |
router-gateway-1-openshift-default |
ScalingReplicaSet |
Scaled down replica set router-gateway-1-openshift-default-799f46c59b from 1 to 0 | |
kserve-ci-e2e-test |
kubelet |
gw-section-name-router-with-gat-f1d92d0f-kserve-5f5b776bbfj4qcm |
FailedPreStopHook |
PreStopHook failed | |
kserve-ci-e2e-test |
kubelet |
router-gateway-1-openshift-default-799f46c59b-gx98l |
Unhealthy |
Readiness probe failed: HTTP probe failed with statuscode: 503 | |
kserve-ci-e2e-test |
replicaset-controller |
router-gateway-1-openshift-default-799f46c59b |
SuccessfulDelete |
Deleted pod: router-gateway-1-openshift-default-799f46c59b-gx98l | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
autoscale-hpa-deploy |
Created |
Created v1.Service kserve-ci-e2e-test/autoscale-hpa-deploy-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
autoscale-hpa-deploy |
Created |
Created v1.Secret kserve-ci-e2e-test/autoscale-hpa-deploy-kserve-self-signed-certs | |
kserve-ci-e2e-test |
deployment-controller |
autoscale-hpa-deploy-kserve |
ScalingReplicaSet |
Scaled up replica set autoscale-hpa-deploy-kserve-66746bd8b9 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
autoscale-hpa-deploy |
Created |
Created v1.Deployment kserve-ci-e2e-test/autoscale-hpa-deploy-kserve | |
kserve-ci-e2e-test |
kubelet |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
Created |
Created container: main | |
kserve-ci-e2e-test |
multus |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
AddedInterface |
Add eth0 [10.132.0.33/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
replicaset-controller |
autoscale-hpa-deploy-kserve-66746bd8b9 |
SuccessfulCreate |
Created pod: autoscale-hpa-deploy-kserve-66746bd8b9-f297r | |
kserve-ci-e2e-test |
kubelet |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
Pulled |
Container image "ghcr.io/llm-d/llm-d-inference-sim-dev:1d5ad96" already present on machine | |
kserve-ci-e2e-test |
kubelet |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
Started |
Started container main | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
autoscale-hpa-deploy |
Updated |
Updated v1.Secret kserve-ci-e2e-test/autoscale-hpa-deploy-kserve-self-signed-certs | |
| (x12) | kserve-ci-e2e-test |
LLMInferenceServiceController |
autoscale-hpa-deploy |
Error |
Reconciliation failed: failed to reconcile workload: failed to reconcile scaling: failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling kserve-ci-e2e-test/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1" |
| (x13) | kserve-ci-e2e-test |
OpenDataHubModelController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route" not found |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-sa | |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve |
ScalingReplicaSet |
Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" already present on machine | |
kserve-ci-e2e-test |
multus |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
AddedInterface |
Add eth0 [10.133.0.59/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-rb | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-role | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-workload-svc | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984 |
SuccessfulCreate |
Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-6c6dcbfc95 |
SuccessfulCreate |
Created pod: llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx | |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-scheduler-6c6dcbfc95 from 0 to 1 | |
kserve-ci-e2e-test |
multus |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
AddedInterface |
Add eth0 [10.132.0.34/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-epp-service | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Pulling |
Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-model-fb-opt-125m-with-7ca60146, llmisvc-router-managed-test-llm-4b931143 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, llmisvc-model-fb-opt-125m-with-7ca60146, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test, llmisvc-model-fb-opt-125m-with-7ca60146, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-concurrency-test; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-model-fb-opt-125m-with-7ca60146, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143; safe to ignore if traffic splitting is planned | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Updated |
Updated v1.Secret kserve-ci-e2e-test/llmisv3e414c2ba058a022dfd694dbcbac5b51-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-route | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146-kserve-shadow-svc |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Pulled |
Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" in 27.978s (27.978s including waiting). Image size: 3531177328 bytes. | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Created |
Created container: main | |
| (x7) | kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Unhealthy |
Startup probe failed: Get "https://10.133.0.59:8000/health": dial tcp 10.133.0.59:8000: connect: connection refused |
| (x7) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-7ca60146; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
LLMInferenceServiceReady |
LLMInferenceService [llmisvc-model-fb-opt-125m-with-7ca60146] is Ready | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-7ca60146 |
Deleted |
Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test5216bfd716f919dc046bc693ceb22e41-llmisvc-mode | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Killing |
Stopping container main | |
| (x5) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned |
| (x10) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143; safe to ignore if traffic splitting is planned |
| (x5) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-concurrency-test; safe to ignore if traffic splitting is planned |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-router-managed-test-llm-4b931143 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15; safe to ignore if traffic splitting is planned |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-router-schem8sfx |
Unhealthy |
Readiness probe failed: service unhealthy (responded with "NOT_SERVING") | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-7ca60146-kserve-74d68bf984-2dn6z |
Unhealthy |
Readiness probe failed: Get "https://10.133.0.59:8000/health": dial tcp 10.133.0.59:8000: connect: connection refused | |
| (x10) | kserve-ci-e2e-test |
OpenDataHubModelController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
ReconcileError |
Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route" not found |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve |
ScalingReplicaSet |
Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.ClusterRoleBinding /kserve-ci-e2e-test05addb65ba05195619f26ef266e8fc04-llmisvc-mode | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-workload-svc | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-sa | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c |
SuccessfulCreate |
Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" already present on machine | |
kserve-ci-e2e-test |
multus |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
AddedInterface |
Add eth0 [10.132.0.35/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-role | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
multus |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
AddedInterface |
Add eth0 [10.133.0.60/23] from ovn-kubernetes | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-service | |
kserve-ci-e2e-test |
replicaset-controller |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-7885885764 |
SuccessfulCreate |
Created pod: llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g | |
kserve-ci-e2e-test |
deployment-controller |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler |
ScalingReplicaSet |
Scaled up replica set llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler-7885885764 from 0 to 1 | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-epp-rb | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-scheduler | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Pulled |
Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:735ced4d2d869072184c156c722fd9f0956b6fb71ca85783529cf832d9acbf7b" already present on machine | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Created |
Created container: storage-initializer | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Started |
Started container storage-initializer | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-router-schewz48g |
Pulled |
Container image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker:v0.9.0" already present on machine | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Started |
Started container main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Created |
Created container: main | |
kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Pulled |
Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.19.0" already present on machine | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Updated |
Updated v1.Secret kserve-ci-e2e-test/llmisv77ff2528d3e9b4972cd9335229fce9f0-kserve-self-signed-certs | |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Updated |
Updated v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-route | |
| (x6) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
Created |
(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-shadow-svc |
| (x7) | kserve-ci-e2e-test |
kubelet |
llmisvc-model-fb-opt-125m-with-ba4d693a-kserve-9bddccd4c-z5w8r |
Unhealthy |
Startup probe failed: Get "https://10.133.0.60:8000/health": dial tcp 10.133.0.60:8000: connect: connection refused |
kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
LLMInferenceServiceReady |
LLMInferenceService [llmisvc-model-fb-opt-125m-with-ba4d693a] is Ready | |
| (x20) | kserve-ci-e2e-test |
LLMInferenceServiceController |
autoscale-hpa-deploy |
ScalingCRDNotFound |
Required scaling CRD not installed: failed to reconcile main workload scaling: failed to reconcile main VA: failed to get v1alpha1.VariantAutoscaling kserve-ci-e2e-test/autoscale-hpa-deploy-kserve-va: no matches for kind "VariantAutoscaling" in version "llmd.ai/v1alpha1" |
kserve-ci-e2e-test |
kubelet |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
Killing |
Stopping container main | |
kserve-ci-e2e-test |
kubelet |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
Unhealthy |
Readiness probe failed: Get "https://10.132.0.33:8000/health": dial tcp 10.132.0.33:8000: connect: connection refused | |
kserve-ci-e2e-test |
kubelet |
autoscale-hpa-deploy-kserve-66746bd8b9-f297r |
FailedPreStopHook |
PreStopHook failed | |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-smoke-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, llmisvc-model-fb-opt-125m-with-ba4d693a, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-smoke-test; safe to ignore if traffic splitting is planned |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceController |
fc-concurrency-test |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-smoke-test, llmisvc-model-fb-opt-125m-with-ba4d693a, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/fc-concurrency-test; safe to ignore if traffic splitting is planned |
| (x2) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-4b931143 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-model-fb-opt-125m-with-ba4d693a, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-4b931143; safe to ignore if traffic splitting is planned |
| (x3) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-router-managed-test-llm-5b1e8f15 |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-model-fb-opt-125m-with-ba4d693a, llmisvc-router-managed-test-llm-4b931143 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-router-managed-test-llm-5b1e8f15; safe to ignore if traffic splitting is planned |
| (x8) | kserve-ci-e2e-test |
LLMInferenceServiceController |
llmisvc-model-fb-opt-125m-with-ba4d693a |
ModelNameCollision |
model name "facebook/opt-125m" overlaps with fc-concurrency-test, fc-smoke-test, llmisvc-router-managed-test-llm-4b931143, llmisvc-router-managed-test-llm-5b1e8f15 in namespace kserve-ci-e2e-test; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-with-ba4d693a; safe to ignore if traffic splitting is planned |