Events

Time	Namespace	Component	RelatedObject	Reason	Message
14:40:09	kserve-ci-e2e-test		llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg to ip-10-0-132-130.ec2.internal
14:29:15	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg to ip-10-0-133-31.ec2.internal
14:27:35	kserve-ci-e2e-test		scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h to ip-10-0-132-130.ec2.internal
14:27:35	kserve-ci-e2e-test		scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq to ip-10-0-133-31.ec2.internal
14:28:52	kserve-ci-e2e-test		scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk to ip-10-0-133-31.ec2.internal
14:28:52	kserve-ci-e2e-test		scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx to ip-10-0-132-130.ec2.internal
14:34:08	kserve-ci-e2e-test		router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Scheduled	Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7 to ip-10-0-132-130.ec2.internal
14:34:08	kserve-ci-e2e-test		router-with-refs-test-kserve-8ff8bc774-bcw5h	Scheduled	Successfully assigned kserve-ci-e2e-test/router-with-refs-test-kserve-8ff8bc774-bcw5h to ip-10-0-133-31.ec2.internal
14:43:38	kserve-ci-e2e-test		scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd to ip-10-0-132-130.ec2.internal
14:43:40	kserve-ci-e2e-test		scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler-8574822pls to ip-10-0-133-31.ec2.internal
14:30:53	kserve-ci-e2e-test		stop-feature-test-kserve-786885549d-4vzv6	Scheduled	Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-786885549d-4vzv6 to ip-10-0-133-31.ec2.internal
14:33:06	kserve-ci-e2e-test		stop-feature-test-kserve-786885549d-8dvrh	Scheduled	Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-786885549d-8dvrh to ip-10-0-133-31.ec2.internal
14:44:08	kserve-ci-e2e-test		router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Scheduled	Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k to ip-10-0-132-130.ec2.internal
14:33:06	kserve-ci-e2e-test		stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Scheduled	Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q to ip-10-0-132-130.ec2.internal
14:30:48	kserve-ci-e2e-test		conv-test-round-trip-kserve-6469f58555-gl6gv	Scheduled	Successfully assigned kserve-ci-e2e-test/conv-test-round-trip-kserve-6469f58555-gl6gv to ip-10-0-133-31.ec2.internal
14:30:53	kserve-ci-e2e-test		stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Scheduled	Successfully assigned kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf to ip-10-0-132-130.ec2.internal
14:40:25	kserve-ci-e2e-test		custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Scheduled	Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-c58f8844b-v8blc to ip-10-0-133-31.ec2.internal
14:40:09	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1 to ip-10-0-132-130.ec2.internal
14:43:34	kserve-ci-e2e-test		router-gateway-2-openshift-default-6866b85949-jzrxz	Scheduled	Successfully assigned kserve-ci-e2e-test/router-gateway-2-openshift-default-6866b85949-jzrxz to ip-10-0-133-31.ec2.internal
14:44:08	kserve-ci-e2e-test		router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Scheduled	Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-64bf496d54-bxxzr to ip-10-0-132-130.ec2.internal
14:29:16	kserve-ci-e2e-test		llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm to ip-10-0-132-130.ec2.internal
14:23:18	openshift-ingress		istiod-openshift-gateway-7cd77c7ffd-l8rhj	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-7cd77c7ffd-l8rhj to ip-10-0-133-31.ec2.internal
14:23:47	kuadrant-system		dns-operator-controller-manager-844548ff4c-qf7lt	Scheduled	Successfully assigned kuadrant-system/dns-operator-controller-manager-844548ff4c-qf7lt to ip-10-0-136-18.ec2.internal
14:28:51	kserve-ci-e2e-test		scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Scheduled	Successfully assigned kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx to ip-10-0-132-130.ec2.internal
14:29:52	kserve-ci-e2e-test		precise-prefix-cache-test-kserve-646bf96947-25kbf	Scheduled	Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-646bf96947-25kbf to ip-10-0-132-130.ec2.internal
14:40:09	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 to ip-10-0-133-31.ec2.internal
14:27:46	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c to ip-10-0-133-31.ec2.internal
14:27:19	kserve-ci-e2e-test		router-gateway-1-openshift-default-6c59fbf55c-k9fch	Scheduled	Successfully assigned kserve-ci-e2e-test/router-gateway-1-openshift-default-6c59fbf55c-k9fch to ip-10-0-133-31.ec2.internal
14:31:55	kserve-ci-e2e-test		custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Scheduled	Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp to ip-10-0-132-130.ec2.internal
14:36:27	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc to ip-10-0-133-31.ec2.internal
14:23:52	kuadrant-system		limitador-operator-controller-manager-c7fb4c8d5-l56ml	Scheduled	Successfully assigned kuadrant-system/limitador-operator-controller-manager-c7fb4c8d5-l56ml to ip-10-0-136-18.ec2.internal
14:40:25	kserve-ci-e2e-test		custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Scheduled	Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq to ip-10-0-133-31.ec2.internal
14:17:18	openshift-ingress		router-default-56d64c8c4-q4z5h	Scheduled	Successfully assigned openshift-ingress/router-default-56d64c8c4-q4z5h to ip-10-0-132-130.ec2.internal
14:24:03	kuadrant-system		kuadrant-console-plugin-6c886788f8-zmhlt	Scheduled	Successfully assigned kuadrant-system/kuadrant-console-plugin-6c886788f8-zmhlt to ip-10-0-132-130.ec2.internal
14:36:16	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f to ip-10-0-133-31.ec2.internal
14:24:45	kuadrant-system		limitador-limitador-67566c68b4-85cgp	Scheduled	Successfully assigned kuadrant-system/limitador-limitador-67566c68b4-85cgp to ip-10-0-133-31.ec2.internal
14:23:50	kuadrant-system		authorino-operator-7587b89b76-dcrzg	Scheduled	Successfully assigned kuadrant-system/authorino-operator-7587b89b76-dcrzg to ip-10-0-133-31.ec2.internal
14:31:54	kserve-ci-e2e-test		custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Scheduled	Successfully assigned kserve-ci-e2e-test/custom-route-timeout-test-kserve-7ffd45c99d-wfq2q to ip-10-0-133-31.ec2.internal
14:25:21	openshift-ingress		istiod-openshift-gateway-55ff986f96-7cccv	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-55ff986f96-7cccv to ip-10-0-136-18.ec2.internal
14:24:46	kuadrant-system		authorino-79cbc94b89-hnk7k	Scheduled	Successfully assigned kuadrant-system/authorino-79cbc94b89-hnk7k to ip-10-0-132-130.ec2.internal
14:25:11	kuadrant-system		authorino-68bd676465-jr972	Scheduled	Successfully assigned kuadrant-system/authorino-68bd676465-jr972 to ip-10-0-132-130.ec2.internal
14:24:46	kuadrant-system		authorino-674b59b84c-qzb68	Scheduled	Successfully assigned kuadrant-system/authorino-674b59b84c-qzb68 to ip-10-0-132-130.ec2.internal
14:23:56	kuadrant-system		kuadrant-operator-controller-manager-6ddf9554fc-fhq85	Scheduled	Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-6ddf9554fc-fhq85 to ip-10-0-133-31.ec2.internal
14:23:25	openshift-ingress		openshift-ai-inference-openshift-default-7c5447bb76-vs482	Scheduled	Successfully assigned openshift-ingress/openshift-ai-inference-openshift-default-7c5447bb76-vs482 to ip-10-0-133-31.ec2.internal
14:40:26	kserve-ci-e2e-test		custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Scheduled	Successfully assigned kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr to ip-10-0-132-130.ec2.internal
14:44:08	kserve-ci-e2e-test		router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Scheduled	Successfully assigned kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl to ip-10-0-132-130.ec2.internal
14:36:16	kserve-ci-e2e-test		llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx to ip-10-0-133-31.ec2.internal
14:24:45	kuadrant-system		limitador-limitador-64c8f475fb-hc7j2	Scheduled	Successfully assigned kuadrant-system/limitador-limitador-64c8f475fb-hc7j2 to ip-10-0-136-18.ec2.internal
14:36:16	kserve-ci-e2e-test		llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Scheduled	Successfully assigned kserve-ci-e2e-test/llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk to ip-10-0-132-130.ec2.internal
14:29:52	kserve-ci-e2e-test		precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Scheduled	Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x to ip-10-0-133-31.ec2.internal
14:29:52	kserve-ci-e2e-test		precise-prefix-cache-test-kserve-646bf96947-5wkrj	Scheduled	Successfully assigned kserve-ci-e2e-test/precise-prefix-cache-test-kserve-646bf96947-5wkrj to ip-10-0-133-31.ec2.internal
14:10:56	openshift-ingress	deployment-controller	router-default	ScalingReplicaSet	Scaled up replica set router-default-56d64c8c4 from 0 to 1
14:11:13	openshift-ingress	service-controller	router-default	EnsuringLoadBalancer	Ensuring load balancer
14:11:13	openshift-ingress	service-controller	router-default	UnAvailableLoadBalancer	There are no available nodes for LoadBalancer
14:11:17	openshift-ingress	service-controller	router-default	EnsuredLoadBalancer	Ensured load balancer
14:11:17 (x13)	openshift-ingress	replicaset-controller	router-default-56d64c8c4	FailedCreate	Error creating: pods "router-default-56d64c8c4-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
14:14:34 (x16)	openshift-ingress	replicaset-controller	router-default-56d64c8c4	FailedCreate	Error creating: pods "router-default-56d64c8c4-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
14:15:00 (x3)	openshift-ingress	service-controller	router-default	UpdatedLoadBalancer	Updated load balancer with new hosts
14:17:18	openshift-ingress	replicaset-controller	router-default-56d64c8c4	SuccessfulCreate	Created pod: router-default-56d64c8c4-q4z5h
14:17:25 (x5)	openshift-ingress	kubelet	router-default-56d64c8c4-q4z5h	FailedMount	MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
14:17:25 (x5)	openshift-ingress	kubelet	router-default-56d64c8c4-q4z5h	FailedMount	MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
14:17:34	openshift-ingress	kubelet	router-default-56d64c8c4-q4z5h	Pulled	Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:294af5c64228434d1ed6ee8ea3ac802e3c999aa847223e3b2efa18425a9fe421" already present on machine
14:17:34	openshift-ingress	kubelet	router-default-56d64c8c4-q4z5h	Created	Created container: router
14:17:34	openshift-ingress	kubelet	router-default-56d64c8c4-q4z5h	Started	Started container router
14:17:34	openshift-ingress	multus	router-default-56d64c8c4-q4z5h	AddedInterface	Add eth0 [10.134.0.8/23] from ovn-kubernetes
14:23:18	openshift-ingress	multus	istiod-openshift-gateway-7cd77c7ffd-l8rhj	AddedInterface	Add eth0 [10.133.0.21/23] from ovn-kubernetes
14:23:18	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8"
14:23:18	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-7cd77c7ffd from 0 to 1
14:23:18	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7cd77c7ffd	SuccessfulCreate	Created pod: istiod-openshift-gateway-7cd77c7ffd-l8rhj
14:23:21	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	Started	Started container discovery
14:23:21	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8" in 2.267s (2.267s including waiting). Image size: 179322965 bytes.
14:23:21	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	Created	Created container: discovery
14:23:22	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	ProbeError	Readiness probe error: HTTP probe failed with statuscode: 503 body:
14:23:22	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	Unhealthy	Readiness probe failed: HTTP probe failed with statuscode: 503
14:23:25	openshift-ingress	kubelet	openshift-ai-inference-openshift-default-7c5447bb76-vs482	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe"
14:23:25	openshift-ingress	service-controller	openshift-ai-inference-openshift-default	EnsuringLoadBalancer	Ensuring load balancer
14:23:25	openshift-ingress	gateway_labeler_controller	openshift-ai-inference	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway openshift-ai-inference
14:23:25	openshift-ingress	deployment-controller	openshift-ai-inference-openshift-default	ScalingReplicaSet	Scaled up replica set openshift-ai-inference-openshift-default-7c5447bb76 from 0 to 1
14:23:25	openshift-ingress	replicaset-controller	openshift-ai-inference-openshift-default-7c5447bb76	SuccessfulCreate	Created pod: openshift-ai-inference-openshift-default-7c5447bb76-vs482
14:23:25	openshift-ingress	multus	openshift-ai-inference-openshift-default-7c5447bb76-vs482	AddedInterface	Add eth0 [10.133.0.22/23] from ovn-kubernetes
14:23:28	openshift-ingress	service-controller	openshift-ai-inference-openshift-default	EnsuredLoadBalancer	Ensured load balancer
14:23:28	openshift-ingress	kubelet	openshift-ai-inference-openshift-default-7c5447bb76-vs482	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" in 2.537s (2.537s including waiting). Image size: 163191192 bytes.
14:23:28	openshift-ingress	kubelet	openshift-ai-inference-openshift-default-7c5447bb76-vs482	Created	Created container: istio-proxy
14:23:28	openshift-ingress	kubelet	openshift-ai-inference-openshift-default-7c5447bb76-vs482	Started	Started container istio-proxy
14:23:44	kuadrant-system	operator-lifecycle-manager	dns-operator.v1.3.0	RequirementsUnknown	requirements not yet checked
14:23:44	kuadrant-system	operator-lifecycle-manager	dns-operator.v1.3.0	RequirementsNotMet	one or more requirements couldn't be found
14:23:47	kuadrant-system	operator-lifecycle-manager	authorino-operator.v1.3.0	RequirementsUnknown	requirements not yet checked
14:23:47	kuadrant-system	deployment-controller	dns-operator-controller-manager	ScalingReplicaSet	Scaled up replica set dns-operator-controller-manager-844548ff4c from 0 to 1
14:23:47	kuadrant-system	operator-lifecycle-manager	dns-operator.v1.3.0	InstallWaiting	installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.
14:23:47	kuadrant-system	operator-lifecycle-manager	authorino-operator.v1.3.0	RequirementsNotMet	one or more requirements couldn't be found
14:23:47	kuadrant-system	operator-lifecycle-manager	dns-operator.v1.3.0	AllRequirementsMet	all requirements found, attempting install
14:23:47	kuadrant-system	replicaset-controller	dns-operator-controller-manager-844548ff4c	SuccessfulCreate	Created pod: dns-operator-controller-manager-844548ff4c-qf7lt
14:23:47	kuadrant-system	operator-lifecycle-manager	dns-operator.v1.3.0	InstallSucceeded	waiting for install components to report healthy
14:23:48	kuadrant-system	multus	dns-operator-controller-manager-844548ff4c-qf7lt	AddedInterface	Add eth0 [10.132.0.29/23] from ovn-kubernetes
14:23:48	kuadrant-system	kubelet	dns-operator-controller-manager-844548ff4c-qf7lt	Pulling	Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:b4e7ba67509320ca9ac5d63cc4add987fad05b098c4a7cd8dd91f264731177cf"
14:23:50	kuadrant-system	multus	authorino-operator-7587b89b76-dcrzg	AddedInterface	Add eth0 [10.133.0.27/23] from ovn-kubernetes
14:23:50	kuadrant-system	kubelet	dns-operator-controller-manager-844548ff4c-qf7lt	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:b4e7ba67509320ca9ac5d63cc4add987fad05b098c4a7cd8dd91f264731177cf" in 2.495s (2.495s including waiting). Image size: 186790172 bytes.
14:23:50	kuadrant-system	kubelet	dns-operator-controller-manager-844548ff4c-qf7lt	Created	Created container: manager
14:23:50	kuadrant-system	kubelet	dns-operator-controller-manager-844548ff4c-qf7lt	Started	Started container manager
14:23:50	kuadrant-system	operator-lifecycle-manager	limitador-operator.v1.3.0	RequirementsUnknown	requirements not yet checked
14:23:50	kuadrant-system	operator-lifecycle-manager	limitador-operator.v1.3.0	RequirementsNotMet	one or more requirements couldn't be found
14:23:50	kuadrant-system	operator-lifecycle-manager	authorino-operator.v1.3.0	InstallWaiting	installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.
14:23:50	kuadrant-system	operator-lifecycle-manager	authorino-operator.v1.3.0	InstallSucceeded	waiting for install components to report healthy
14:23:50	kuadrant-system	operator-lifecycle-manager	authorino-operator.v1.3.0	AllRequirementsMet	all requirements found, attempting install
14:23:50	kuadrant-system	deployment-controller	authorino-operator	ScalingReplicaSet	Scaled up replica set authorino-operator-7587b89b76 from 0 to 1
14:23:50	kuadrant-system	replicaset-controller	authorino-operator-7587b89b76	SuccessfulCreate	Created pod: authorino-operator-7587b89b76-dcrzg
14:23:51	kuadrant-system	dns-operator-controller-manager-844548ff4c-qf7lt_1b9c595d-0638-4c81-8103-f589ee89483a	a3f98d6c.kuadrant.io	LeaderElection	dns-operator-controller-manager-844548ff4c-qf7lt_1b9c595d-0638-4c81-8103-f589ee89483a became leader
14:23:51	kuadrant-system	kubelet	authorino-operator-7587b89b76-dcrzg	Pulling	Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:4e581c0172549df72c6ef8978e96d771645bc36298d2ccf9240af4e71c088733"
14:23:52	kuadrant-system	operator-lifecycle-manager	limitador-operator.v1.3.0	InstallSucceeded	waiting for install components to report healthy
14:23:52	kuadrant-system	operator-lifecycle-manager	limitador-operator.v1.3.0	AllRequirementsMet	all requirements found, attempting install
14:23:52	kuadrant-system	replicaset-controller	limitador-operator-controller-manager-c7fb4c8d5	SuccessfulCreate	Created pod: limitador-operator-controller-manager-c7fb4c8d5-l56ml
14:23:52	kuadrant-system	deployment-controller	limitador-operator-controller-manager	ScalingReplicaSet	Scaled up replica set limitador-operator-controller-manager-c7fb4c8d5 from 0 to 1
14:23:53	kuadrant-system	operator-lifecycle-manager	limitador-operator.v1.3.0	InstallWaiting	installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.
14:23:53	kuadrant-system	authorino-operator-7587b89b76-dcrzg_ab254bc2-e663-4ee2-9bc8-91440f70638d	aac3a15d.authorino.kuadrant.io	LeaderElection	authorino-operator-7587b89b76-dcrzg_ab254bc2-e663-4ee2-9bc8-91440f70638d became leader
14:23:53	kuadrant-system	kubelet	authorino-operator-7587b89b76-dcrzg	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:4e581c0172549df72c6ef8978e96d771645bc36298d2ccf9240af4e71c088733" in 2.388s (2.388s including waiting). Image size: 161991975 bytes.
14:23:53	kuadrant-system	kubelet	authorino-operator-7587b89b76-dcrzg	Created	Created container: manager
14:23:53	kuadrant-system	operator-lifecycle-manager	rhcl-operator.v1.3.2	RequirementsUnknown	requirements not yet checked
14:23:53	kuadrant-system	kubelet	authorino-operator-7587b89b76-dcrzg	Started	Started container manager
14:23:53	kuadrant-system	multus	limitador-operator-controller-manager-c7fb4c8d5-l56ml	AddedInterface	Add eth0 [10.132.0.30/23] from ovn-kubernetes
14:23:53	kuadrant-system	kubelet	limitador-operator-controller-manager-c7fb4c8d5-l56ml	Pulling	Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:9a3ccb54bc904521277b87e1d178d8134cc6ef744eb4ec5b64177355fd5a9e66"
14:23:53	kuadrant-system	operator-lifecycle-manager	rhcl-operator.v1.3.2	RequirementsNotMet	one or more requirements couldn't be found
14:23:54	kuadrant-system	kubelet	limitador-operator-controller-manager-c7fb4c8d5-l56ml	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:9a3ccb54bc904521277b87e1d178d8134cc6ef744eb4ec5b64177355fd5a9e66" in 1.653s (1.653s including waiting). Image size: 175506011 bytes.
14:23:55	kuadrant-system	limitador-operator-controller-manager-c7fb4c8d5-l56ml_5fd352b0-ae09-4e6d-bd61-ec85a92659e3	3745a16e.kuadrant.io	LeaderElection	limitador-operator-controller-manager-c7fb4c8d5-l56ml_5fd352b0-ae09-4e6d-bd61-ec85a92659e3 became leader
14:23:55	kuadrant-system	kubelet	limitador-operator-controller-manager-c7fb4c8d5-l56ml	Created	Created container: manager
14:23:55	kuadrant-system	kubelet	limitador-operator-controller-manager-c7fb4c8d5-l56ml	Started	Started container manager
14:23:55	kuadrant-system	operator-lifecycle-manager	rhcl-operator.v1.3.2	AllRequirementsMet	all requirements found, attempting install
14:23:56	kuadrant-system	kubelet	kuadrant-operator-controller-manager-6ddf9554fc-fhq85	Pulling	Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:7aa6d6dbbde488260789adb94fa232a06d1e635cc2aaf86af24f8cf14a17eef8"
14:23:56	kuadrant-system	deployment-controller	kuadrant-operator-controller-manager	ScalingReplicaSet	Scaled up replica set kuadrant-operator-controller-manager-6ddf9554fc from 0 to 1
14:23:56	kuadrant-system	operator-lifecycle-manager	rhcl-operator.v1.3.2	InstallWaiting	installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
14:23:56	kuadrant-system	operator-lifecycle-manager	rhcl-operator.v1.3.2	InstallSucceeded	waiting for install components to report healthy
14:23:56	kuadrant-system	multus	kuadrant-operator-controller-manager-6ddf9554fc-fhq85	AddedInterface	Add eth0 [10.133.0.28/23] from ovn-kubernetes
14:23:56	kuadrant-system	replicaset-controller	kuadrant-operator-controller-manager-6ddf9554fc	SuccessfulCreate	Created pod: kuadrant-operator-controller-manager-6ddf9554fc-fhq85
14:24:00	kuadrant-system	kubelet	kuadrant-operator-controller-manager-6ddf9554fc-fhq85	Started	Started container manager
14:24:00	kuadrant-system	kubelet	kuadrant-operator-controller-manager-6ddf9554fc-fhq85	Created	Created container: manager
14:24:00	kuadrant-system	kubelet	kuadrant-operator-controller-manager-6ddf9554fc-fhq85	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:7aa6d6dbbde488260789adb94fa232a06d1e635cc2aaf86af24f8cf14a17eef8" in 3.886s (3.886s including waiting). Image size: 411782238 bytes.
14:24:00	kuadrant-system	kuadrant-operator-controller-manager-6ddf9554fc-fhq85_86bb9466-21af-44f0-9ba7-b15625615f2f	f139389e.kuadrant.io	LeaderElection	kuadrant-operator-controller-manager-6ddf9554fc-fhq85_86bb9466-21af-44f0-9ba7-b15625615f2f became leader
14:24:02	kuadrant-system	operator-lifecycle-manager	dns-operator.v1.3.0	InstallSucceeded	install strategy completed with no errors
14:24:03	kuadrant-system	deployment-controller	kuadrant-console-plugin	ScalingReplicaSet	Scaled up replica set kuadrant-console-plugin-6c886788f8 from 0 to 1
14:24:03	kuadrant-system	replicaset-controller	kuadrant-console-plugin-6c886788f8	SuccessfulCreate	Created pod: kuadrant-console-plugin-6c886788f8-zmhlt
14:24:03	kuadrant-system	kubelet	kuadrant-console-plugin-6c886788f8-zmhlt	FailedMount	MountVolume.SetUp failed for volume "plugin-serving-cert" : secret "plugin-serving-cert" not found
14:24:04	kuadrant-system	kubelet	kuadrant-console-plugin-6c886788f8-zmhlt	Pulling	Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:41755d5e16ee5409cfe8888cf5883b262820baea4eae449fcf61f06ab1756f57"
14:24:04	kuadrant-system	multus	kuadrant-console-plugin-6c886788f8-zmhlt	AddedInterface	Add eth0 [10.134.0.29/23] from ovn-kubernetes
14:24:04	kuadrant-system	operator-lifecycle-manager	authorino-operator.v1.3.0	InstallSucceeded	install strategy completed with no errors
14:24:06	kuadrant-system	operator-lifecycle-manager	limitador-operator.v1.3.0	InstallSucceeded	install strategy completed with no errors
14:24:08	kuadrant-system	kubelet	kuadrant-console-plugin-6c886788f8-zmhlt	Started	Started container kuadrant-console-plugin
14:24:08	kuadrant-system	kubelet	kuadrant-console-plugin-6c886788f8-zmhlt	Created	Created container: kuadrant-console-plugin
14:24:08	kuadrant-system	kubelet	kuadrant-console-plugin-6c886788f8-zmhlt	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:41755d5e16ee5409cfe8888cf5883b262820baea4eae449fcf61f06ab1756f57" in 4.722s (4.722s including waiting). Image size: 339541674 bytes.
14:24:12	kuadrant-system	operator-lifecycle-manager	rhcl-operator.v1.3.2	InstallSucceeded	install strategy completed with no errors
14:24:45	kuadrant-system	replicaset-controller	limitador-limitador-67566c68b4	SuccessfulCreate	Created pod: limitador-limitador-67566c68b4-85cgp
14:24:45	kuadrant-system	deployment-controller	limitador-limitador	ScalingReplicaSet	Scaled up replica set limitador-limitador-67566c68b4 from 0 to 1
14:24:45	kuadrant-system	replicaset-controller	limitador-limitador-64c8f475fb	SuccessfulCreate	Created pod: limitador-limitador-64c8f475fb-hc7j2
14:24:45	kuadrant-system	deployment-controller	limitador-limitador	ScalingReplicaSet	Scaled up replica set limitador-limitador-64c8f475fb from 0 to 1
14:24:45	kuadrant-system	kubelet	limitador-limitador-64c8f475fb-hc7j2	Pulling	Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0"
14:24:45	kuadrant-system	multus	limitador-limitador-64c8f475fb-hc7j2	AddedInterface	Add eth0 [10.132.0.31/23] from ovn-kubernetes
14:24:46	kuadrant-system	deployment-controller	authorino	ScalingReplicaSet	Scaled up replica set authorino-79cbc94b89 from 0 to 1
14:24:46	kuadrant-system	multus	limitador-limitador-67566c68b4-85cgp	AddedInterface	Add eth0 [10.133.0.29/23] from ovn-kubernetes
14:24:46	kuadrant-system	multus	authorino-674b59b84c-qzb68	AddedInterface	Add eth0 [10.134.0.30/23] from ovn-kubernetes
14:24:46	kuadrant-system	replicaset-controller	authorino-674b59b84c	SuccessfulCreate	Created pod: authorino-674b59b84c-qzb68
14:24:46	kuadrant-system	kubelet	limitador-limitador-67566c68b4-85cgp	Pulling	Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0"
14:24:46	kuadrant-system	replicaset-controller	authorino-79cbc94b89	SuccessfulCreate	Created pod: authorino-79cbc94b89-hnk7k
14:24:46	kuadrant-system	deployment-controller	authorino	ScalingReplicaSet	Scaled up replica set authorino-674b59b84c from 0 to 1
14:24:46	kuadrant-system	kubelet	authorino-674b59b84c-qzb68	Pulling	Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"
14:24:47	kuadrant-system	kubelet	authorino-79cbc94b89-hnk7k	Pulling	Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"
14:24:47	kuadrant-system	multus	authorino-79cbc94b89-hnk7k	AddedInterface	Add eth0 [10.134.0.31/23] from ovn-kubernetes
14:24:49	kuadrant-system	kubelet	authorino-79cbc94b89-hnk7k	Created	Created container: authorino
14:24:49	kuadrant-system	kubelet	authorino-79cbc94b89-hnk7k	Started	Started container authorino
14:24:49	kuadrant-system	deployment-controller	authorino	ScalingReplicaSet	Scaled down replica set authorino-674b59b84c from 1 to 0
14:24:49	kuadrant-system	kubelet	limitador-limitador-64c8f475fb-hc7j2	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0" in 3.77s (3.77s including waiting). Image size: 249943843 bytes.
14:24:49	kuadrant-system	replicaset-controller	authorino-674b59b84c	SuccessfulDelete	Deleted pod: authorino-674b59b84c-qzb68
14:24:49	kuadrant-system	kubelet	limitador-limitador-64c8f475fb-hc7j2	Started	Started container limitador
14:24:49	kuadrant-system	kubelet	limitador-limitador-64c8f475fb-hc7j2	Created	Created container: limitador
14:24:49	kuadrant-system	kubelet	authorino-79cbc94b89-hnk7k	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 2.729s (2.729s including waiting). Image size: 199256829 bytes.
14:24:50	kuadrant-system	kubelet	limitador-limitador-67566c68b4-85cgp	Created	Created container: limitador
14:24:50	kuadrant-system	kubelet	limitador-limitador-67566c68b4-85cgp	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:43bc4021bb30c7ef305e01c11e22ca5584c7e2c192b4db0a5646f04f715ee3b0" in 3.987s (3.987s including waiting). Image size: 249943843 bytes.
14:24:50	kuadrant-system	kubelet	limitador-limitador-67566c68b4-85cgp	Started	Started container limitador
14:24:51	kuadrant-system	kubelet	authorino-674b59b84c-qzb68	Created	Created container: authorino
14:24:51	kuadrant-system	kubelet	authorino-674b59b84c-qzb68	Started	Started container authorino
14:24:51	kuadrant-system	kubelet	authorino-674b59b84c-qzb68	Killing	Stopping container authorino
14:24:51	kuadrant-system	kubelet	authorino-674b59b84c-qzb68	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 4.24s (4.24s including waiting). Image size: 199256829 bytes.
14:25:01	kuadrant-system	deployment-controller	limitador-limitador	ScalingReplicaSet	Scaled down replica set limitador-limitador-64c8f475fb from 1 to 0
14:25:01	kuadrant-system	kubelet	limitador-limitador-64c8f475fb-hc7j2	Killing	Stopping container limitador
14:25:01	kuadrant-system	replicaset-controller	limitador-limitador-64c8f475fb	SuccessfulDelete	Deleted pod: limitador-limitador-64c8f475fb-hc7j2
14:25:11	kuadrant-system	multus	authorino-68bd676465-jr972	AddedInterface	Add eth0 [10.134.0.32/23] from ovn-kubernetes
14:25:11	kuadrant-system	deployment-controller	authorino	ScalingReplicaSet	Scaled up replica set authorino-68bd676465 from 0 to 1
14:25:11	kuadrant-system	replicaset-controller	authorino-68bd676465	SuccessfulCreate	Created pod: authorino-68bd676465-jr972
14:25:12	kuadrant-system	kubelet	authorino-68bd676465-jr972	Pulled	Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8" in 444ms (444ms including waiting). Image size: 199256829 bytes.
14:25:12	kuadrant-system	kubelet	authorino-68bd676465-jr972	Created	Created container: authorino
14:25:12	kuadrant-system	kubelet	authorino-68bd676465-jr972	Started	Started container authorino
14:25:12	kuadrant-system	kubelet	authorino-68bd676465-jr972	Pulling	Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c2208382e16c501e4ed58aea83f54d108567f36853cf79f7ffdf8ee54ff4ace8"
14:25:13	kuadrant-system	deployment-controller	authorino	ScalingReplicaSet	Scaled down replica set authorino-79cbc94b89 from 1 to 0
14:25:13	kuadrant-system	replicaset-controller	authorino-79cbc94b89	SuccessfulDelete	Deleted pod: authorino-79cbc94b89-hnk7k
14:25:13	kuadrant-system	kubelet	authorino-79cbc94b89-hnk7k	Killing	Stopping container authorino
14:25:21	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-55ff986f96 from 0 to 1
14:25:21	openshift-ingress	replicaset-controller	istiod-openshift-gateway-55ff986f96	SuccessfulCreate	Created pod: istiod-openshift-gateway-55ff986f96-7cccv
14:25:22	openshift-ingress	multus	istiod-openshift-gateway-55ff986f96-7cccv	AddedInterface	Add eth0 [10.132.0.32/23] from ovn-kubernetes
14:25:22	openshift-ingress	kubelet	istiod-openshift-gateway-55ff986f96-7cccv	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8"
14:25:24	openshift-ingress	kubelet	istiod-openshift-gateway-55ff986f96-7cccv	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:028e10651db0d1ddb769a27c9483c6d41be6ac597f253afd9d599f395d9c82d8" in 2.664s (2.664s including waiting). Image size: 179322965 bytes.
14:25:24	openshift-ingress	kubelet	istiod-openshift-gateway-55ff986f96-7cccv	Created	Created container: discovery
14:25:25	openshift-ingress	kubelet	istiod-openshift-gateway-55ff986f96-7cccv	ProbeError	Readiness probe error: Get "http://10.132.0.32:8080/ready": dial tcp 10.132.0.32:8080: connect: connection refused body:
14:25:25	openshift-ingress	kubelet	istiod-openshift-gateway-55ff986f96-7cccv	Unhealthy	Readiness probe failed: Get "http://10.132.0.32:8080/ready": dial tcp 10.132.0.32:8080: connect: connection refused
14:25:25	openshift-ingress	kubelet	istiod-openshift-gateway-55ff986f96-7cccv	Started	Started container discovery
14:25:26	openshift-ingress	kubelet	istiod-openshift-gateway-7cd77c7ffd-l8rhj	Killing	Stopping container discovery
14:25:26	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled down replica set istiod-openshift-gateway-7cd77c7ffd from 1 to 0
14:25:26	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7cd77c7ffd	SuccessfulDelete	Deleted pod: istiod-openshift-gateway-7cd77c7ffd-l8rhj
14:25:33 (x2)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
14:25:33 (x2)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
14:25:48 (x2)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
14:25:48 (x2)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
14:27:19	kserve-ci-e2e-test	replicaset-controller	router-gateway-1-openshift-default-6c59fbf55c	SuccessfulCreate	Created pod: router-gateway-1-openshift-default-6c59fbf55c-k9fch
14:27:19	kserve-ci-e2e-test	deployment-controller	router-gateway-1-openshift-default	ScalingReplicaSet	Scaled up replica set router-gateway-1-openshift-default-6c59fbf55c from 0 to 1
14:27:19	kserve-ci-e2e-test	service-controller	router-gateway-1-openshift-default	EnsuringLoadBalancer	Ensuring load balancer
14:27:20	kserve-ci-e2e-test	kubelet	router-gateway-1-openshift-default-6c59fbf55c-k9fch	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" already present on machine
14:27:20	kserve-ci-e2e-test	kubelet	router-gateway-1-openshift-default-6c59fbf55c-k9fch	Created	Created container: istio-proxy
14:27:20	kserve-ci-e2e-test	kubelet	router-gateway-1-openshift-default-6c59fbf55c-k9fch	Started	Started container istio-proxy
14:27:20	kserve-ci-e2e-test	multus	router-gateway-1-openshift-default-6c59fbf55c-k9fch	AddedInterface	Add eth0 [10.133.0.37/23] from ovn-kubernetes
14:27:23	kserve-ci-e2e-test	service-controller	router-gateway-1-openshift-default	EnsuredLoadBalancer	Ensured load balancer
14:27:30 (x12)	kserve-ci-e2e-test	OpenDataHubModelController	scheduler-configmap-ref-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-configmap-ref-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-configmap-ref-test-kserve-route" not found
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-rb
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.Role kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-role
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-sa
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-workload-svc
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb
14:27:35	kserve-ci-e2e-test	deployment-controller	scheduler-configmap-ref-test-kserve	ScalingReplicaSet	Scaled up replica set scheduler-configmap-ref-test-kserve-58f94bc4f9 from 0 to 1
14:27:35	kserve-ci-e2e-test	deployment-controller	scheduler-configmap-ref-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set scheduler-configmap-ref-test-kserve-router-scheduler-79cdcb97f7 from 0 to 1
14:27:35	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Pulling	Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d"
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.Deployment kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-router-scheduler
14:27:35	kserve-ci-e2e-test	replicaset-controller	scheduler-configmap-ref-test-kserve-router-scheduler-79cdcb97f7	SuccessfulCreate	Created pod: scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h
14:27:35	kserve-ci-e2e-test	replicaset-controller	scheduler-configmap-ref-test-kserve-58f94bc4f9	SuccessfulCreate	Created pod: scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq
14:27:35	kserve-ci-e2e-test	multus	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	AddedInterface	Add eth0 [10.133.0.38/23] from ovn-kubernetes
14:27:35	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	Created v1.Service kserve-ci-e2e-test/scheduler-configmap-ref-test-epp-service
14:27:36 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-shadow-svc
14:27:37	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	FailedMount	MountVolume.SetUp failed for volume "kube-api-access-rcx6z" : failed to sync configmap cache: timed out waiting for the condition
14:27:38	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Pulling	Pulling image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d"
14:27:38	kserve-ci-e2e-test	multus	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	AddedInterface	Add eth0 [10.134.0.33/23] from ovn-kubernetes
14:27:40	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Started	Started container storage-initializer
14:27:40	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Created	Created container: storage-initializer
14:27:40 (x14)	kserve-ci-e2e-test	OpenDataHubModelController	llmisvc-model-fb-opt-125m-route-6fa1027a	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route" not found
14:27:40	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Pulled	Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" in 4.174s (4.174s including waiting). Image size: 299663288 bytes.
14:27:41	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Pulled	Successfully pulled image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" in 3.314s (3.314s including waiting). Image size: 299663288 bytes.
14:27:41	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Started	Started container storage-initializer
14:27:41	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Created	Created container: storage-initializer
14:27:42	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Pulling	Pulling image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1"
14:27:43	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Started	Started container main
14:27:43	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Pulling	Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1"
14:27:43	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" in 1.027s (1.027s including waiting). Image size: 87241021 bytes.
14:27:43	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Created	Created container: main
14:27:44	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Pulling	Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"
14:27:45	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.185s (1.185s including waiting). Image size: 98346788 bytes.
14:27:45	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Created	Created container: main
14:27:45	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Started	Started container main
14:27:46	kserve-ci-e2e-test	replicaset-controller	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c	SuccessfulCreate	Created pod: llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c
14:27:46	kserve-ci-e2e-test	deployment-controller	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve	ScalingReplicaSet	Scaled up replica set llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c from 0 to 1
14:27:46	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-6fa1027a	Created	Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-workload-svc
14:27:46	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-6fa1027a	Created	Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-route
14:27:46	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-6fa1027a	Created	Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-workload-svc
14:27:46	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-6fa1027a	Created	Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-6fa1027a-kserve
14:27:46	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-6fa1027a	Created	Created v1.Secret kserve-ci-e2e-test/llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs
14:27:47	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	AddedInterface	Add eth0 [10.133.0.39/23] from ovn-kubernetes
14:27:47	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:27:47	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Created	Created container: storage-initializer
14:27:47	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Started	Started container storage-initializer
14:27:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Updated	Updated v1.Secret kserve-ci-e2e-test/scheduler-configmap-ref-test-kserve-self-signed-certs
14:27:52	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Created	Created container: main
14:27:52	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Started	Started container main
14:27:52	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine
14:28:01	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-6fa1027a	Updated	Updated v1.Secret kserve-ci-e2e-test/llmisvad71fa5348b85aebd404221bba611457-kserve-self-signed-certs
14:28:13	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Started	Started container tokenizer
14:28:13	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Created	Created container: tokenizer
14:28:13	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" in 30.209s (30.209s including waiting). Image size: 2907020949 bytes.
14:28:28	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-configmap-ref-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-configmap-ref-test-epp-auth-rb
14:28:29	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-58f94bc4f9-pfsbq	Killing	Stopping container main
14:28:29	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Killing	Stopping container tokenizer
14:28:29	kserve-ci-e2e-test	kubelet	scheduler-configmap-ref-test-kserve-router-scheduler-79cdchjh7h	Killing	Stopping container main
14:28:49 (x13)	kserve-ci-e2e-test	OpenDataHubModelController	scheduler-ha-replicas-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-ha-replicas-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-ha-replicas-test-kserve-route" not found
14:28:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-workload-svc
14:28:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve
14:28:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.Role kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-role
14:28:51	kserve-ci-e2e-test	replicaset-controller	scheduler-ha-replicas-test-kserve-6857dbdbf5	SuccessfulCreate	Created pod: scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx
14:28:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb
14:28:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-sa
14:28:51	kserve-ci-e2e-test	deployment-controller	scheduler-ha-replicas-test-kserve	ScalingReplicaSet	Scaled up replica set scheduler-ha-replicas-test-kserve-6857dbdbf5 from 0 to 1
14:28:51	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Created	Created container: storage-initializer
14:28:52	kserve-ci-e2e-test	multus	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	AddedInterface	Add eth0 [10.134.0.34/23] from ovn-kubernetes
14:28:52	kserve-ci-e2e-test	replicaset-controller	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b88d7	SuccessfulCreate	Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx
14:28:52	kserve-ci-e2e-test	replicaset-controller	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b88d7	SuccessfulCreate	Created pod: scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Started	Started container storage-initializer
14:28:52	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.Service kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-service
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:28:52	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/scheduler-ha-replicas-test-epp-rb
14:28:52 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-shadow-svc
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Created	Created container: storage-initializer
14:28:52	kserve-ci-e2e-test	deployment-controller	scheduler-ha-replicas-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set scheduler-ha-replicas-test-kserve-router-scheduler-7b879b88d7 from 0 to 2
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:28:52	kserve-ci-e2e-test	multus	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	AddedInterface	Add eth0 [10.133.0.40/23] from ovn-kubernetes
14:28:52	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Created	Created v1.Deployment kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-router-scheduler
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Started	Started container storage-initializer
14:28:52	kserve-ci-e2e-test	multus	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	AddedInterface	Add eth0 [10.134.0.35/23] from ovn-kubernetes
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Created	Created container: storage-initializer
14:28:52	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Started	Started container storage-initializer
14:28:53	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-6fa1027a-kserve-55bf56b4c-vdw6c	Killing	Stopping container main
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Created	Created container: main
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Pulling	Pulling image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1"
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Started	Started container main
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Created	Created container: tokenizer
14:28:53	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Started	Started container tokenizer
14:28:54	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Started	Started container main
14:28:54	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Pulling	Pulling image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1"
14:28:54	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Created	Created container: main
14:28:54	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" in 944ms (944ms including waiting). Image size: 87241021 bytes.
14:28:57	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Pulling	Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"
14:28:58	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.198s (1.198s including waiting). Image size: 98346788 bytes.
14:28:59	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Started	Started container main
14:28:59	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Created	Created container: main
14:29:12 (x15)	kserve-ci-e2e-test	OpenDataHubModelController	llmisvc-model-fb-opt-125m-route-55f7ae4a	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-route" not found
14:29:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-sa
14:29:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.Secret kserve-ci-e2e-test/llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs
14:29:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve
14:29:15	kserve-ci-e2e-test	replicaset-controller	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988	SuccessfulCreate	Created pod: llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg
14:29:15	kserve-ci-e2e-test	deployment-controller	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve	ScalingReplicaSet	Scaled up replica set llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988 from 0 to 1
14:29:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-workload-svc
14:29:16	kserve-ci-e2e-test	replicaset-controller	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler-59c6fd4888	SuccessfulCreate	Created pod: llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm
14:29:16 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-shadow-svc
14:29:16	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Started	Started container storage-initializer
14:29:16	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Created	Created container: storage-initializer
14:29:16	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:29:16	kserve-ci-e2e-test	multus	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	AddedInterface	Add eth0 [10.134.0.36/23] from ovn-kubernetes
14:29:16	kserve-ci-e2e-test	deployment-controller	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler-59c6fd4888 from 0 to 1
14:29:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-teste2c0c7deeb42f8b9846a7a45d2caecd2-llmisvc-mode
14:29:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-role
14:29:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-rb
14:29:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.Deployment kserve-ci-e2e-test/llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-scheduler
14:29:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Created	Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-55f7ae4a-epp-service
14:29:17	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:29:17	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Created	Created container: main
14:29:17	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:29:17	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Started	Started container main
14:29:18	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Started	Started container tokenizer
14:29:18	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Created	Created container: tokenizer
14:29:23	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Started	Started container tokenizer
14:29:23	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:29:23	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Created	Created container: tokenizer
14:29:23	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" in 28.946s (28.946s including waiting). Image size: 2907020949 bytes.
14:29:23	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Started	Started container storage-initializer
14:29:23	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Created	Created container: storage-initializer
14:29:23	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	AddedInterface	Add eth0 [10.133.0.41/23] from ovn-kubernetes
14:29:28	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Updated	Updated v1.Secret kserve-ci-e2e-test/scheduler-ha-replicas-test-kserve-self-signed-certs
14:29:28	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Pulling	Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1"
14:29:32	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Unhealthy	Startup probe failed: Get "http://10.133.0.40:8082/healthz": dial tcp 10.133.0.40:8082: connect: connection refused
14:29:38	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Killing	Stopping container main
14:29:38	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Killing	Stopping container tokenizer
14:29:38	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-6857dbdbf5-gv4xx	Killing	Stopping container main
14:29:38	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b8zw8dx	Killing	Stopping container tokenizer
14:29:38	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-ha-replicas-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-ha-replicas-test-epp-auth-rb
14:29:38	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Updated	Updated v1.Secret kserve-ci-e2e-test/llmisvdde380eaa9fe1facad32d45131f9e34d-kserve-self-signed-certs
14:29:38 (x4)	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Unhealthy	Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
14:29:38	kserve-ci-e2e-test	kubelet	scheduler-ha-replicas-test-kserve-router-scheduler-7b879b872clk	Killing	Stopping container main
14:29:39 (x13)	kserve-ci-e2e-test	OpenDataHubModelController	precise-prefix-cache-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy precise-prefix-cache-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/precise-prefix-cache-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "precise-prefix-cache-test-kserve-route" not found
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-kserve-workload-svc
14:29:52	kserve-ci-e2e-test	deployment-controller	precise-prefix-cache-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set precise-prefix-cache-test-kserve-router-scheduler-58f499f669 from 0 to 1
14:29:52	kserve-ci-e2e-test	deployment-controller	precise-prefix-cache-test-kserve	ScalingReplicaSet	Scaled up replica set precise-prefix-cache-test-kserve-646bf96947 from 0 to 2
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/precise-prefix-cache-test-epp-sa
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.Role kserve-ci-e2e-test/precise-prefix-cache-test-epp-role
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/precise-prefix-cache-test-epp-rb
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.Deployment kserve-ci-e2e-test/precise-prefix-cache-test-kserve-router-scheduler
14:29:52	kserve-ci-e2e-test	replicaset-controller	precise-prefix-cache-test-kserve-646bf96947	SuccessfulCreate	Created pod: precise-prefix-cache-test-kserve-646bf96947-5wkrj
14:29:52	kserve-ci-e2e-test	replicaset-controller	precise-prefix-cache-test-kserve-646bf96947	SuccessfulCreate	Created pod: precise-prefix-cache-test-kserve-646bf96947-25kbf
14:29:52	kserve-ci-e2e-test	replicaset-controller	precise-prefix-cache-test-kserve-router-scheduler-58f499f669	SuccessfulCreate	Created pod: precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x
14:29:52	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	Created v1.Service kserve-ci-e2e-test/precise-prefix-cache-test-epp-service
14:29:53	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:29:53	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Started	Started container storage-initializer
14:29:53 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/precise-prefix-cache-test-kserve-shadow-svc
14:29:53	kserve-ci-e2e-test	multus	precise-prefix-cache-test-kserve-646bf96947-25kbf	AddedInterface	Add eth0 [10.134.0.37/23] from ovn-kubernetes
14:29:53	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Created	Created container: storage-initializer
14:29:56	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:29:56	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Started	Started container storage-initializer
14:29:56	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Pulled	Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" in 28.089s (28.089s including waiting). Image size: 3458932386 bytes.
14:29:56	kserve-ci-e2e-test	multus	precise-prefix-cache-test-kserve-646bf96947-5wkrj	AddedInterface	Add eth0 [10.133.0.42/23] from ovn-kubernetes
14:29:56	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:29:56	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Created	Created container: storage-initializer
14:29:56	kserve-ci-e2e-test	multus	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	AddedInterface	Add eth0 [10.133.0.43/23] from ovn-kubernetes
14:29:56	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Started	Started container storage-initializer
14:29:56	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Created	Created container: storage-initializer
14:29:57	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Started	Started container main
14:29:57	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Created	Created container: main
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Started	Started container tokenizer
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Created	Created container: main
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Started	Started container main
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Created	Created container: main
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Started	Started container main
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:29:58	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Created	Created container: tokenizer
14:30:02	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine
14:30:02	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Created	Created container: main
14:30:02	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Started	Started container main
14:30:03	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Unhealthy	Startup probe failed: Get "http://10.133.0.43:8082/healthz": dial tcp 10.133.0.43:8082: connect: connection refused
14:30:07 (x2)	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Updated	Updated v1.Secret kserve-ci-e2e-test/precise-prefix-cache-test-kserve-self-signed-certs
14:30:14 (x2)	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Unhealthy	Liveness probe failed: timeout: failed to connect service "10.133.0.43:9003" within 1s: context deadline exceeded
14:30:35	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Killing	Stopping container main
14:30:35	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-router-scheduler-58f499f6d725x	Killing	Stopping container tokenizer
14:30:35	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-5wkrj	Killing	Stopping container main
14:30:35	kserve-ci-e2e-test	LLMInferenceServiceController	precise-prefix-cache-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-precise-prefix-cache-test-epp-auth-rb
14:30:35	kserve-ci-e2e-test	kubelet	precise-prefix-cache-test-kserve-646bf96947-25kbf	Killing	Stopping container main
14:30:48	kserve-ci-e2e-test	LLMInferenceServiceController	conv-test-round-trip	Created	Created v1.DestinationRule kserve-ci-e2e-test/conv-test-round-trip-kserve-workload-svc
14:30:48	kserve-ci-e2e-test	LLMInferenceServiceController	conv-test-round-trip	Created	Created v1.Secret kserve-ci-e2e-test/conv-test-round-trip-kserve-self-signed-certs
14:30:48	kserve-ci-e2e-test	multus	conv-test-round-trip-kserve-6469f58555-gl6gv	AddedInterface	Add eth0 [10.133.0.44/23] from ovn-kubernetes
14:30:48	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:30:48	kserve-ci-e2e-test	deployment-controller	conv-test-round-trip-kserve	ScalingReplicaSet	Scaled up replica set conv-test-round-trip-kserve-6469f58555 from 0 to 1
14:30:48	kserve-ci-e2e-test	replicaset-controller	conv-test-round-trip-kserve-6469f58555	SuccessfulCreate	Created pod: conv-test-round-trip-kserve-6469f58555-gl6gv
14:30:48	kserve-ci-e2e-test	LLMInferenceServiceController	conv-test-round-trip	Created	Created v1.Service kserve-ci-e2e-test/conv-test-round-trip-kserve-workload-svc
14:30:48	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Created	Created container: storage-initializer
14:30:48	kserve-ci-e2e-test	LLMInferenceServiceController	conv-test-round-trip	Created	Created v1.Deployment kserve-ci-e2e-test/conv-test-round-trip-kserve
14:30:48	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Started	Started container storage-initializer
14:30:48	kserve-ci-e2e-test	LLMInferenceServiceController	conv-test-round-trip	Created	Created v1.HTTPRoute kserve-ci-e2e-test/conv-test-round-trip-kserve-route
14:30:53	kserve-ci-e2e-test	deployment-controller	stop-feature-test-kserve	ScalingReplicaSet	Scaled up replica set stop-feature-test-kserve-786885549d from 0 to 1
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/stop-feature-test-epp-rb
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.Role kserve-ci-e2e-test/stop-feature-test-epp-role
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-stop-feature-test-epp-auth-rb
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa
14:30:53	kserve-ci-e2e-test	replicaset-controller	stop-feature-test-kserve-786885549d	SuccessfulCreate	Created pod: stop-feature-test-kserve-786885549d-4vzv6
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve
14:30:53	kserve-ci-e2e-test	replicaset-controller	stop-feature-test-kserve-router-scheduler-57d5754c4	SuccessfulCreate	Created pod: stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf
14:30:53	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	Created v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service
14:30:53	kserve-ci-e2e-test	deployment-controller	stop-feature-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set stop-feature-test-kserve-router-scheduler-57d5754c4 from 0 to 1
14:30:54 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/stop-feature-test-kserve-shadow-svc
14:30:54	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Created	Created container: storage-initializer
14:30:54	kserve-ci-e2e-test	multus	stop-feature-test-kserve-786885549d-4vzv6	AddedInterface	Add eth0 [10.133.0.45/23] from ovn-kubernetes
14:30:54	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:30:54	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Started	Started container storage-initializer
14:30:54	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Started	Started container storage-initializer
14:30:54	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Created	Created container: main
14:30:54	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Started	Started container main
14:30:54	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:30:54	kserve-ci-e2e-test	multus	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	AddedInterface	Add eth0 [10.134.0.38/23] from ovn-kubernetes
14:30:54	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:30:54	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Created	Created container: storage-initializer
14:30:56	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Created	Created container: main
14:30:56	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Started	Started container tokenizer
14:30:56	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Created	Created container: tokenizer
14:30:56	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:30:56	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Started	Started container main
14:30:56	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:30:57	kserve-ci-e2e-test	kubelet	conv-test-round-trip-kserve-6469f58555-gl6gv	Killing	Stopping container main
14:30:59	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Started	Started container main
14:30:59	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Created	Created container: main
14:30:59	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:31:14	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Updated	Updated v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs
14:31:26 (x9)	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Unhealthy	Startup probe failed: Get "https://10.133.0.41:8000/health": dial tcp 10.133.0.41:8000: connect: connection refused
14:31:42	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-55f7ae4a	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-teste2c0c7deeb42f8b9846a7a45d2caecd2-llmisvc-mode
14:31:43	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Killing	Stopping container tokenizer
14:31:43	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-55f7ae4a-kserve-766c677988b2kqg	Killing	Stopping container main
14:31:43	kserve-ci-e2e-test	kubelet	llmisvcdde380eaa9fe1facad32d45131f9e34d-kserve-router-schel9zpm	Killing	Stopping container main
14:31:53 (x12)	kserve-ci-e2e-test	OpenDataHubModelController	custom-route-timeout-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-test-kserve-route" not found
14:31:54	kserve-ci-e2e-test	deployment-controller	custom-route-timeout-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set custom-route-timeout-test-kserve-router-scheduler-977d94f48 from 0 to 1
14:31:54	kserve-ci-e2e-test	deployment-controller	custom-route-timeout-test-kserve	ScalingReplicaSet	Scaled up replica set custom-route-timeout-test-kserve-7ffd45c99d from 0 to 1
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-test-epp-sa
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-kserve-workload-svc
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.Role kserve-ci-e2e-test/custom-route-timeout-test-epp-role
14:31:54	kserve-ci-e2e-test	replicaset-controller	custom-route-timeout-test-kserve-7ffd45c99d	SuccessfulCreate	Created pod: custom-route-timeout-test-kserve-7ffd45c99d-wfq2q
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.Service kserve-ci-e2e-test/custom-route-timeout-test-epp-service
14:31:54	kserve-ci-e2e-test	replicaset-controller	custom-route-timeout-test-kserve-router-scheduler-977d94f48	SuccessfulCreate	Created pod: custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-test-epp-rb
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs
14:31:54	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-test-kserve-router-scheduler
14:31:55 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-test-kserve-shadow-svc
14:31:55	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:31:55	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Created	Created container: storage-initializer
14:31:55	kserve-ci-e2e-test	multus	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	AddedInterface	Add eth0 [10.134.0.39/23] from ovn-kubernetes
14:31:55	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:31:55	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Started	Started container storage-initializer
14:31:55	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Started	Started container storage-initializer
14:31:55	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Created	Created container: storage-initializer
14:31:55	kserve-ci-e2e-test	multus	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	AddedInterface	Add eth0 [10.133.0.46/23] from ovn-kubernetes
14:31:56	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Created	Created container: tokenizer
14:31:56	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Created	Created container: main
14:31:56	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Started	Started container main
14:31:56	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:31:56	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Started	Started container tokenizer
14:31:56	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:32:00	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:32:00	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Created	Created container: main
14:32:00	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Started	Started container main
14:32:06	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Updated	Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-test-kserve-self-signed-certs
14:32:34 (x10)	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Unhealthy	Startup probe failed: Get "https://10.133.0.45:8000/health": dial tcp 10.133.0.45:8000: connect: connection refused
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route
14:32:45	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-4vzv6	Killing	Stopping container main
14:32:45	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Killing	Stopping container tokenizer
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1alpha2.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.InferencePool kserve-ci-e2e-test/stop-feature-test-inference-pool
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-epp-service
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve-router-scheduler
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.ServiceAccount kserve-ci-e2e-test/stop-feature-test-epp-sa
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.Service kserve-ci-e2e-test/stop-feature-test-kserve-workload-svc
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.Deployment kserve-ci-e2e-test/stop-feature-test-kserve
14:32:45	kserve-ci-e2e-test	LLMInferenceServiceController	stop-feature-test	Deleted	Deleted v1.Secret kserve-ci-e2e-test/stop-feature-test-kserve-self-signed-certs
14:32:45	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-q7hcf	Killing	Stopping container main
14:32:53 (x25)	kserve-ci-e2e-test	OpenDataHubModelController	stop-feature-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy stop-feature-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/stop-feature-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "stop-feature-test-kserve-route" not found
14:33:06	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Started	Started container storage-initializer
14:33:06	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Started	Started container storage-initializer
14:33:06	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Created	Created container: storage-initializer
14:33:06	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:33:06	kserve-ci-e2e-test	multus	stop-feature-test-kserve-786885549d-8dvrh	AddedInterface	Add eth0 [10.133.0.47/23] from ovn-kubernetes
14:33:06	kserve-ci-e2e-test	multus	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	AddedInterface	Add eth0 [10.134.0.40/23] from ovn-kubernetes
14:33:06	kserve-ci-e2e-test	replicaset-controller	stop-feature-test-kserve-786885549d	SuccessfulCreate	Created pod: stop-feature-test-kserve-786885549d-8dvrh
14:33:06	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:33:06	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Created	Created container: storage-initializer
14:33:06	kserve-ci-e2e-test	replicaset-controller	stop-feature-test-kserve-router-scheduler-57d5754c4	SuccessfulCreate	Created pod: stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q
14:33:06	kserve-ci-e2e-test	deployment-controller	stop-feature-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set stop-feature-test-kserve-router-scheduler-57d5754c4 from 0 to 1
14:33:06	kserve-ci-e2e-test	deployment-controller	stop-feature-test-kserve	ScalingReplicaSet	Scaled up replica set stop-feature-test-kserve-786885549d from 0 to 1
14:33:07	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:33:07	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Started	Started container tokenizer
14:33:07	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:33:07	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Created	Created container: main
14:33:07	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Started	Started container main
14:33:07	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Created	Created container: tokenizer
14:33:11	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:33:11	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Started	Started container main
14:33:11	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Created	Created container: main
14:33:25 (x9)	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Unhealthy	Startup probe failed: Get "https://10.133.0.46:8000/health": dial tcp 10.133.0.46:8000: connect: connection refused
14:33:40 (x2)	kserve-ci-e2e-test	gateway_labeler_controller	router-gateway-1	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway router-gateway-1
14:33:40	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-test-epp-auth-rb
14:33:40	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Killing	Stopping container tokenizer
14:33:40	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Killing	Stopping container main
14:33:40	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-7ffd45c99d-wfq2q	Killing	Stopping container main
14:33:45	kserve-ci-e2e-test	kubelet	custom-route-timeout-test-kserve-router-scheduler-977d94f455fcp	Unhealthy	Readiness probe failed: Get "http://10.134.0.39:8082/healthz": dial tcp 10.134.0.39:8082: connect: connection refused
14:34:08	kserve-ci-e2e-test	deployment-controller	router-with-refs-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set router-with-refs-test-kserve-router-scheduler-9b96b9744 from 0 to 1
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.Service kserve-ci-e2e-test/router-with-refs-test-epp-service
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve-router-scheduler
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-test-epp-rb
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.Role kserve-ci-e2e-test/router-with-refs-test-epp-role
14:34:08	kserve-ci-e2e-test	replicaset-controller	router-with-refs-test-kserve-router-scheduler-9b96b9744	SuccessfulCreate	Created pod: router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-test-epp-sa
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.Service kserve-ci-e2e-test/router-with-refs-test-kserve-workload-svc
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.Deployment kserve-ci-e2e-test/router-with-refs-test-kserve
14:34:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	Created v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs
14:34:08	kserve-ci-e2e-test	deployment-controller	router-with-refs-test-kserve	ScalingReplicaSet	Scaled up replica set router-with-refs-test-kserve-8ff8bc774 from 0 to 1
14:34:08	kserve-ci-e2e-test	multus	router-with-refs-test-kserve-8ff8bc774-bcw5h	AddedInterface	Add eth0 [10.133.0.48/23] from ovn-kubernetes
14:34:08	kserve-ci-e2e-test	replicaset-controller	router-with-refs-test-kserve-8ff8bc774	SuccessfulCreate	Created pod: router-with-refs-test-kserve-8ff8bc774-bcw5h
14:34:08	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:34:08	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Created	Created container: storage-initializer
14:34:08	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Started	Started container storage-initializer
14:34:09	kserve-ci-e2e-test	multus	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	AddedInterface	Add eth0 [10.134.0.41/23] from ovn-kubernetes
14:34:09 (x5)	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-test-kserve-shadow-svc
14:34:09	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Started	Started container storage-initializer
14:34:09	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Created	Created container: storage-initializer
14:34:09	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:34:10	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Created	Created container: main
14:34:10	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:34:10	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:34:10	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Started	Started container main
14:34:11	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Created	Created container: tokenizer
14:34:11	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Started	Started container tokenizer
14:34:13	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Created	Created container: main
14:34:13	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Started	Started container main
14:34:13	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:34:16	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Updated	Updated v1.Secret kserve-ci-e2e-test/router-with-refs-test-kserve-self-signed-certs
14:34:46 (x10)	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Unhealthy	Startup probe failed: Get "https://10.133.0.47:8000/health": dial tcp 10.133.0.47:8000: connect: connection refused
14:34:57	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Killing	Stopping container tokenizer
14:34:57	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Killing	Stopping container main
14:34:58	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-router-scheduler-57d5754c4-8c96q	Unhealthy	Readiness probe failed: timeout: failed to connect service "10.134.0.40:9003" within 1s: context deadline exceeded
14:34:58	kserve-ci-e2e-test	kubelet	stop-feature-test-kserve-786885549d-8dvrh	Killing	Stopping container main
14:35:38 (x9)	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Unhealthy	Startup probe failed: Get "https://10.133.0.48:8000/health": dial tcp 10.133.0.48:8000: connect: connection refused
14:36:00	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-8ff8bc774-bcw5h	Killing	Stopping container main
14:36:00	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Killing	Stopping container main
14:36:00	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Killing	Stopping container tokenizer
14:36:00	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-test-epp-auth-rb
14:36:01	kserve-ci-e2e-test	kubelet	router-with-refs-test-kserve-router-scheduler-9b96b9744-clcz7	Unhealthy	Readiness probe failed: timeout: failed to connect service "10.134.0.41:9003" within 1s: context deadline exceeded
14:36:10 (x12)	kserve-ci-e2e-test	OpenDataHubModelController	llmisvc-model-fb-opt-125m-route-05aa9bba	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-route" not found
14:36:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-role
14:36:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve
14:36:15	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.Secret kserve-ci-e2e-test/llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs
14:36:16	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	AddedInterface	Add eth0 [10.133.0.49/23] from ovn-kubernetes
14:36:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-workload-svc
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Pulling	Pulling image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1"
14:36:16	kserve-ci-e2e-test	deployment-controller	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler-5f8b67d698 from 0 to 1
14:36:16	kserve-ci-e2e-test	replicaset-controller	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheduler-5f8b67d698	SuccessfulCreate	Created pod: llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Started	Started container storage-initializer
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Created	Created container: storage-initializer
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:36:16	kserve-ci-e2e-test	multus	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	AddedInterface	Add eth0 [10.134.0.42/23] from ovn-kubernetes
14:36:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test4ce8108fb3d8aff064c322bee6e173f5-llmisvc-mode
14:36:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-epp-sa
14:36:16	kserve-ci-e2e-test	replicaset-controller	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487	SuccessfulCreate	Created pod: llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f
14:36:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill
14:36:16	kserve-ci-e2e-test	replicaset-controller	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-594548454	SuccessfulCreate	Created pod: llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx
14:36:16	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	AddedInterface	Add eth0 [10.133.0.50/23] from ovn-kubernetes
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Created	Created container: storage-initializer
14:36:16	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Started	Started container storage-initializer
14:36:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve
14:36:16	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-rb
14:36:16	kserve-ci-e2e-test	deployment-controller	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve	ScalingReplicaSet	Scaled up replica set llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487 from 0 to 1
14:36:16	kserve-ci-e2e-test	deployment-controller	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill	ScalingReplicaSet	Scaled up replica set llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-594548454 from 0 to 1
14:36:17 (x10)	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-shadow-svc
14:36:17	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" in 1.08s (1.08s including waiting). Image size: 67767940 bytes.
14:36:17	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Created	Created container: llm-d-routing-sidecar
14:36:17 (x13)	kserve-ci-e2e-test	OpenDataHubModelController	llmisvc-model-fb-opt-125m-route-02ac88e7	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route" not found
14:36:17	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Started	Started container llm-d-routing-sidecar
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Started	Started container tokenizer
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Started	Started container main
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Created	Created container: tokenizer
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Created	Created container: main
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Created	Created container: storage-initializer
14:36:18	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Started	Started container storage-initializer
14:36:21	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Started	Started container main
14:36:21	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Created	Created container: main
14:36:21	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:36:23	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Created	Created container: main
14:36:23	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:36:23	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Started	Started container main
14:36:27	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-02ac88e7	Created	Created v1.HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-route
14:36:27	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-02ac88e7	Created	Created v1.Deployment kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve
14:36:27	kserve-ci-e2e-test	deployment-controller	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve	ScalingReplicaSet	Scaled up replica set llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b from 0 to 1
14:36:27	kserve-ci-e2e-test	replicaset-controller	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b	SuccessfulCreate	Created pod: llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc
14:36:27	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-02ac88e7	Created	Created v1.Secret kserve-ci-e2e-test/llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs
14:36:27	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-02ac88e7	Created	Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-workload-svc
14:36:28	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-02ac88e7	Created	Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-workload-svc
14:36:28	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Created	Created container: storage-initializer
14:36:28	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:36:28	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	AddedInterface	Add eth0 [10.133.0.51/23] from ovn-kubernetes
14:36:28	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Started	Started container storage-initializer
14:36:33	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:36:33	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Started	Started container main
14:36:33	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Created	Created container: main
14:36:39	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Updated	Updated v1.Secret kserve-ci-e2e-test/llmisv8f1a6f044e8c7a4d31a250e0c4861caf-kserve-self-signed-certs
14:36:44	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-02ac88e7	Updated	Updated v1.Secret kserve-ci-e2e-test/llmisv8de1d74aab16d9cabd8b5aafeb5248e8-kserve-self-signed-certs
14:39:06 (x17)	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Unhealthy	Startup probe failed: Get "https://10.133.0.50:8000/health": dial tcp 10.133.0.50:8000: connect: connection refused
14:39:16 (x18)	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Unhealthy	Startup probe failed: Get "https://10.133.0.49:8001/health": dial tcp 10.133.0.49:8001: connect: connection refused
14:39:18 (x17)	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Unhealthy	Startup probe failed: Get "https://10.133.0.51:8000/health": dial tcp 10.133.0.51:8000: connect: connection refused
14:39:36	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-02ac88e7-kserve-8b8dbf45b-cb2tc	Killing	Stopping container main
14:39:57 (x13)	kserve-ci-e2e-test	OpenDataHubModelController	llmisvc-model-fb-opt-125m-route-f312f5ec	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route: HTTPRoute.gateway.networking.k8s.io "llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-route" not found
14:40:09	kserve-ci-e2e-test	replicaset-controller	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler-78d7889598	SuccessfulCreate	Created pod: llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-testc9aefdf01a18c3acb55ce247710ebb71-llmisvc-mode
14:40:09	kserve-ci-e2e-test	deployment-controller	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-scheduler-78d7889598 from 0 to 1
14:40:09	kserve-ci-e2e-test	multus	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	AddedInterface	Add eth0 [10.134.0.44/23] from ovn-kubernetes
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-rb
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.Role kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-role
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-epp-sa
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.Service kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-workload-svc
14:40:09	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	AddedInterface	Add eth0 [10.134.0.43/23] from ovn-kubernetes
14:40:09	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:40:09	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Created	Created container: storage-initializer
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.LeaderWorkerSet kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.ServiceAccount kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn
14:40:09	kserve-ci-e2e-test	statefulset-controller	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	SuccessfulCreate	create Pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1 in StatefulSet llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 successful
14:40:09	kserve-ci-e2e-test	multus	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	AddedInterface	Add eth0 [10.133.0.52/23] from ovn-kubernetes
14:40:09	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:40:09	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Created	Created container: storage-initializer
14:40:09	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Started	Started container storage-initializer
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.RoleBinding kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-scc
14:40:09	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	Created v1.Secret kserve-ci-e2e-test/llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs
14:40:09	kserve-ci-e2e-test	leaderworkerset	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn	GroupsProgressing	Created worker statefulset for leader pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0
14:40:09	kserve-ci-e2e-test	leaderworkerset	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn	CreatingRevision	Creating revision with key 67b7655b58 for a newly created LeaderWorkerSet
14:40:09	kserve-ci-e2e-test	leaderworkerset	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn	GroupsProgressing	Created leader statefulset llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn
14:40:09 (x2)	kserve-ci-e2e-test	leaderworkerset	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn	GroupsProgressing	Replicas are progressing, with 0 groups ready of total 1 groups
14:40:09	kserve-ci-e2e-test	statefulset-controller	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn	SuccessfulCreate	create Pod llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0 in StatefulSet llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn successful
14:40:10 (x8)	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-shadow-svc
14:40:10	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Started	Started container storage-initializer
14:40:10	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Created	Created container: storage-initializer
14:40:10	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:40:10	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Started	Started container storage-initializer
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Killing	Stopping container tokenizer
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:40:11	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-05aa9bba	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test4ce8108fb3d8aff064c322bee6e173f5-llmisvc-mode
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Created	Created container: tokenizer
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-prefill-59kh5wx	Killing	Stopping container main
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Killing	Stopping container main
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Started	Started container tokenizer
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-05aa9bba-kserve-794cdc4487jx24f	Killing	Stopping container llm-d-routing-sidecar
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Started	Started container main
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Created	Created container: main
14:40:11	kserve-ci-e2e-test	kubelet	llmisvc8f1a6f044e8c7a4d31a250e0c4861caf-kserve-router-scheb6phk	Killing	Stopping container main
14:40:15	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Created	Created container: main
14:40:15	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:40:15	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Pulling	Pulling image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1"
14:40:15	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Started	Started container main
14:40:19	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Unhealthy	Startup probe failed: Get "http://10.134.0.44:8082/healthz": dial tcp 10.134.0.44:8082: connect: connection refused
14:40:23 (x15)	kserve-ci-e2e-test	OpenDataHubModelController	custom-route-timeout-pd-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy custom-route-timeout-pd-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "custom-route-timeout-pd-test-kserve-route" not found
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.Role kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-role
14:40:25	kserve-ci-e2e-test	deployment-controller	custom-route-timeout-pd-test-kserve-prefill	ScalingReplicaSet	Scaled up replica set custom-route-timeout-pd-test-kserve-prefill-b9b967f79 from 0 to 1
14:40:25	kserve-ci-e2e-test	deployment-controller	custom-route-timeout-pd-test-kserve	ScalingReplicaSet	Scaled up replica set custom-route-timeout-pd-test-kserve-c58f8844b from 0 to 1
14:40:25	kserve-ci-e2e-test	replicaset-controller	custom-route-timeout-pd-test-kserve-c58f8844b	SuccessfulCreate	Created pod: custom-route-timeout-pd-test-kserve-c58f8844b-v8blc
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-rb
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.Deployment kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-prefill
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/custom-route-timeout-pd-test-epp-sa
14:40:25	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	Created v1.Service kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-workload-svc
14:40:25	kserve-ci-e2e-test	replicaset-controller	custom-route-timeout-pd-test-kserve-prefill-b9b967f79	SuccessfulCreate	Created pod: custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq
14:40:26	kserve-ci-e2e-test	multus	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	AddedInterface	Add eth0 [10.133.0.54/23] from ovn-kubernetes
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:40:26	kserve-ci-e2e-test	replicaset-controller	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc47b677	SuccessfulCreate	Created pod: custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Started	Started container storage-initializer
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Created	Created container: storage-initializer
14:40:26 (x10)	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-shadow-svc
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Started	Started container llm-d-routing-sidecar
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Created	Created container: llm-d-routing-sidecar
14:40:26	kserve-ci-e2e-test	deployment-controller	custom-route-timeout-pd-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set custom-route-timeout-pd-test-kserve-router-scheduler-6cbc47b677 from 0 to 1
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Pulled	Container image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" already present on machine
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Started	Started container storage-initializer
14:40:26	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Created	Created container: storage-initializer
14:40:26	kserve-ci-e2e-test	multus	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	AddedInterface	Add eth0 [10.133.0.53/23] from ovn-kubernetes
14:40:28	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Started	Started container storage-initializer
14:40:28	kserve-ci-e2e-test	multus	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	AddedInterface	Add eth0 [10.134.0.45/23] from ovn-kubernetes
14:40:28	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:40:28	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Created	Created container: storage-initializer
14:40:31	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:40:31	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Created	Created container: main
14:40:31	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Started	Started container main
14:40:32	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Started	Started container main
14:40:32	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Created	Created container: main
14:40:32	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:40:40	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:40:42	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Created	Created container: main
14:40:42	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Started	Started container tokenizer
14:40:42	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Created	Created container: main
14:40:42	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Pulled	Successfully pulled image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" in 27.358s (27.358s including waiting). Image size: 3458932386 bytes.
14:40:42	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Started	Started container main
14:40:42	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:40:42	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Created	Created container: tokenizer
14:40:42	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Started	Started container main
14:40:59	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Updated	Updated v1.Secret kserve-ci-e2e-test/llmisv4e643bc258191ffc517a31cd1d0ddd27-kserve-self-signed-certs
14:41:05	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Updated	Updated v1.Secret kserve-ci-e2e-test/custom-route-timeout-pd-test-kserve-self-signed-certs
14:43:06 (x16)	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Unhealthy	Startup probe failed: Get "https://10.133.0.54:8000/health": dial tcp 10.133.0.54:8000: connect: connection refused
14:43:09 (x18)	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Unhealthy	Startup probe failed: Get "https://10.133.0.52:8000/health": dial tcp 10.133.0.52:8000: connect: connection refused
14:43:16 (x17)	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Unhealthy	Startup probe failed: Get "https://10.133.0.53:8001/health": dial tcp 10.133.0.53:8001: connect: connection refused
14:43:19 (x2)	kserve-ci-e2e-test	leaderworkerset	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn	AllGroupsReady	All replicas are ready, with 1 groups ready of total 1 groups
14:43:27	kserve-ci-e2e-test	LLMInferenceServiceController	llmisvc-model-fb-opt-125m-route-f312f5ec	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-testc9aefdf01a18c3acb55ce247710ebb71-llmisvc-mode
14:43:27	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Killing	Stopping container tokenizer
14:43:27	kserve-ci-e2e-test	kubelet	llmisvc4e643bc258191ffc517a31cd1d0ddd27-kserve-router-schedkxhg	Killing	Stopping container main
14:43:27	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0	Killing	Stopping container main
14:43:29	kserve-ci-e2e-test	kubelet	llmisvc-model-fb-opt-125m-route-f312f5ec-kserve-mn-0-1	Killing	Stopping container main
14:43:34	kserve-ci-e2e-test	gateway_labeler_controller	router-gateway-2	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway router-gateway-2
14:43:34	kserve-ci-e2e-test	deployment-controller	router-gateway-2-openshift-default	ScalingReplicaSet	Scaled up replica set router-gateway-2-openshift-default-6866b85949 from 0 to 1
14:43:34	kserve-ci-e2e-test	replicaset-controller	router-gateway-2-openshift-default-6866b85949	SuccessfulCreate	Created pod: router-gateway-2-openshift-default-6866b85949-jzrxz
14:43:34	kserve-ci-e2e-test	service-controller	router-gateway-2-openshift-default	EnsuringLoadBalancer	Ensuring load balancer
14:43:35	kserve-ci-e2e-test	kubelet	router-gateway-2-openshift-default-6866b85949-jzrxz	Started	Started container istio-proxy
14:43:35	kserve-ci-e2e-test	multus	router-gateway-2-openshift-default-6866b85949-jzrxz	AddedInterface	Add eth0 [10.133.0.55/23] from ovn-kubernetes
14:43:35	kserve-ci-e2e-test	kubelet	router-gateway-2-openshift-default-6866b85949-jzrxz	Created	Created container: istio-proxy
14:43:35	kserve-ci-e2e-test	kubelet	router-gateway-2-openshift-default-6866b85949-jzrxz	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:d518f3d1539f45e1253c5c9fa22062802804601d4998cd50344e476a3cc388fe" already present on machine
14:43:37	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs
14:43:38	kserve-ci-e2e-test	multus	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	AddedInterface	Add eth0 [10.134.0.46/23] from ovn-kubernetes
14:43:38	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve
14:43:38 (x12)	kserve-ci-e2e-test	OpenDataHubModelController	scheduler-inline-config-test	ReconcileError	Failed to reconcile LLMInferenceService: 1 error occurred: * failed to get HTTPRoute for AuthPolicy scheduler-inline-config-test-kserve-route-authn: failed to get HTTPRoute kserve-ci-e2e-test/scheduler-inline-config-test-kserve-route: HTTPRoute.gateway.networking.k8s.io "scheduler-inline-config-test-kserve-route" not found
14:43:38	kserve-ci-e2e-test	deployment-controller	scheduler-inline-config-test-kserve	ScalingReplicaSet	Scaled up replica set scheduler-inline-config-test-kserve-5dd6f9b7ff from 0 to 1
14:43:38	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:43:38	kserve-ci-e2e-test	service-controller	router-gateway-2-openshift-default	EnsuredLoadBalancer	Ensured load balancer
14:43:38	kserve-ci-e2e-test	replicaset-controller	scheduler-inline-config-test-kserve-5dd6f9b7ff	SuccessfulCreate	Created pod: scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd
14:43:38	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Created	Created container: storage-initializer
14:43:38	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Started	Started container storage-initializer
14:43:39	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-kserve-workload-svc
14:43:39	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/scheduler-inline-config-test-epp-sa
14:43:39	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb
14:43:39	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.Role kserve-ci-e2e-test/scheduler-inline-config-test-epp-role
14:43:40	kserve-ci-e2e-test	replicaset-controller	scheduler-inline-config-test-kserve-router-scheduler-857486bb87	SuccessfulCreate	Created pod: scheduler-inline-config-test-kserve-router-scheduler-8574822pls
14:43:40	kserve-ci-e2e-test	deployment-controller	scheduler-inline-config-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set scheduler-inline-config-test-kserve-router-scheduler-857486bb87 from 0 to 1
14:43:40	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/scheduler-inline-config-test-epp-rb
14:43:40	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.Deployment kserve-ci-e2e-test/scheduler-inline-config-test-kserve-router-scheduler
14:43:40	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	Created v1.Service kserve-ci-e2e-test/scheduler-inline-config-test-epp-service
14:43:40	kserve-ci-e2e-test	multus	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	AddedInterface	Add eth0 [10.133.0.56/23] from ovn-kubernetes
14:43:41	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Started	Started container storage-initializer
14:43:41	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Created	Created container: storage-initializer
14:43:41	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:43:41 (x6)	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/scheduler-inline-config-test-kserve-shadow-svc
14:43:42	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:43:42	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Created	Created container: tokenizer
14:43:42	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Started	Started container main
14:43:42	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Started	Started container tokenizer
14:43:42	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Created	Created container: main
14:43:42	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:43:43	kserve-ci-e2e-test	LLMInferenceServiceController	custom-route-timeout-pd-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-custom-route-timeout-pd-test-epp-auth-rb
14:43:43	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" already present on machine
14:43:44	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Killing	Stopping container main
14:43:44	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-router-scheduler-6cbc49t9dr	Killing	Stopping container tokenizer
14:43:44	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-prefill-b9b967f79-rf2cq	Killing	Stopping container main
14:43:44	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Killing	Stopping container main
14:43:44	kserve-ci-e2e-test	kubelet	custom-route-timeout-pd-test-kserve-c58f8844b-v8blc	Killing	Stopping container llm-d-routing-sidecar
14:43:44	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Created	Created container: main
14:43:44	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Started	Started container main
14:44:08	kserve-ci-e2e-test	deployment-controller	router-with-refs-pd-test-kserve	ScalingReplicaSet	Scaled up replica set router-with-refs-pd-test-kserve-64bf496d54 from 0 to 1
14:44:08	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Created	Created container: storage-initializer
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-epp-sa
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb
14:44:08	kserve-ci-e2e-test	deployment-controller	router-with-refs-pd-test-kserve-prefill	ScalingReplicaSet	Scaled up replica set router-with-refs-pd-test-kserve-prefill-5859b66879 from 0 to 1
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve-prefill
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.Deployment kserve-ci-e2e-test/router-with-refs-pd-test-kserve
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.RoleBinding kserve-ci-e2e-test/router-with-refs-pd-test-kserve-rb
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.Role kserve-ci-e2e-test/router-with-refs-pd-test-kserve-role
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.ServiceAccount kserve-ci-e2e-test/router-with-refs-pd-test-kserve
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs
14:44:08	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:44:08	kserve-ci-e2e-test	replicaset-controller	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d9	SuccessfulCreate	Created pod: router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k
14:44:08	kserve-ci-e2e-test	deployment-controller	router-with-refs-pd-test-kserve-router-scheduler	ScalingReplicaSet	Scaled up replica set router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d9 from 0 to 1
14:44:08	kserve-ci-e2e-test	replicaset-controller	router-with-refs-pd-test-kserve-prefill-5859b66879	SuccessfulCreate	Created pod: router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl
14:44:08	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Started	Started container storage-initializer
14:44:08	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	Created v1.Service kserve-ci-e2e-test/router-with-refs-pd-test-kserve-workload-svc
14:44:08	kserve-ci-e2e-test	multus	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	AddedInterface	Add eth0 [10.134.0.47/23] from ovn-kubernetes
14:44:08	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Pulling	Pulling image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1"
14:44:08	kserve-ci-e2e-test	multus	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	AddedInterface	Add eth0 [10.134.0.48/23] from ovn-kubernetes
14:44:08	kserve-ci-e2e-test	replicaset-controller	router-with-refs-pd-test-kserve-64bf496d54	SuccessfulCreate	Created pod: router-with-refs-pd-test-kserve-64bf496d54-bxxzr
14:44:09 (x9)	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Created	(combined from similar events): Created v1.DestinationRule kserve-ci-e2e-test/router-with-refs-pd-test-kserve-shadow-svc
14:44:09	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Started	Started container storage-initializer
14:44:09	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Started	Started container llm-d-routing-sidecar
14:44:09	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Created	Created container: llm-d-routing-sidecar
14:44:09	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Pulled	Successfully pulled image "ghcr.io/llm-d/llm-d-routing-sidecar:v0.7.1" in 1.057s (1.057s including waiting). Image size: 67767940 bytes.
14:44:09	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Created	Created container: storage-initializer
14:44:09	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:44:09	kserve-ci-e2e-test	multus	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	AddedInterface	Add eth0 [10.134.0.49/23] from ovn-kubernetes
14:44:10	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Created	Created container: storage-initializer
14:44:10	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Started	Started container storage-initializer
14:44:10	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Pulled	Container image "quay.io/opendatahub/kserve-storage-initializer@sha256:498563386b50b61380307187059de966bc0a117e7161dd79d36731f0b2ebd43d" already present on machine
14:44:11	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Created	Created container: tokenizer
14:44:11	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Started	Started container tokenizer
14:44:11	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Created	Created container: main
14:44:11	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Pulled	Container image "ghcr.io/llm-d/llm-d-uds-tokenizer:v0.7.1" already present on machine
14:44:11	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Started	Started container main
14:44:11	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Pulled	Container image "ghcr.io/llm-d/llm-d-inference-scheduler:v0.7.1" already present on machine
14:44:14	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:44:14	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Created	Created container: main
14:44:14	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Started	Started container main
14:44:15	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Started	Started container main
14:44:15	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Created	Created container: main
14:44:15	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Pulled	Container image "public.ecr.aws/q9t5s3a7/vllm-cpu-release-repo:v0.17.1" already present on machine
14:44:18	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Updated	Updated v1.Secret kserve-ci-e2e-test/scheduler-inline-config-test-kserve-self-signed-certs
14:44:19	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Unhealthy	Startup probe failed: Get "http://10.134.0.49:8082/healthz": dial tcp 10.134.0.49:8082: connect: connection refused
14:44:21	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Updated	Updated v1.Secret kserve-ci-e2e-test/router-with-refs-pd-test-kserve-self-signed-certs
14:44:22	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Killing	Stopping container main
14:44:22	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Killing	Stopping container tokenizer
14:44:22	kserve-ci-e2e-test	LLMInferenceServiceController	scheduler-inline-config-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-scheduler-inline-config-test-epp-auth-rb
14:44:22	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-5dd6f9b7ff-qrzrd	Killing	Stopping container main
14:44:23	kserve-ci-e2e-test	kubelet	scheduler-inline-config-test-kserve-router-scheduler-8574822pls	Unhealthy	Readiness probe failed: timeout: failed to connect service "10.133.0.56:9003" within 1s: context deadline exceeded
14:46:28 (x14)	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Unhealthy	Startup probe failed: Get "https://10.134.0.48:8000/health": dial tcp 10.134.0.48:8000: connect: connection refused
14:46:28 (x14)	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Unhealthy	Startup probe failed: Get "https://10.134.0.47:8001/health": dial tcp 10.134.0.47:8001: connect: connection refused
14:46:49	kserve-ci-e2e-test	LLMInferenceServiceController	router-with-refs-pd-test	Deleted	Deleted v1.ClusterRoleBinding /kserve-ci-e2e-test-router-with-refs-pd-test-epp-auth-rb
14:46:50	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Killing	Stopping container tokenizer
14:46:50	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-router-scheduler-6ccc9d84d6rr8k	Killing	Stopping container main
14:46:50	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-prefill-5859b66879-9wgcl	Killing	Stopping container main
14:46:50	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Killing	Stopping container llm-d-routing-sidecar
14:46:50	kserve-ci-e2e-test	kubelet	router-with-refs-pd-test-kserve-64bf496d54-bxxzr	Killing	Stopping container main