Events

Time	Namespace	Component	RelatedObject	Reason	Message
00:56:32	openshift-ingress		payload-processing-f9b8d9b9b-x5m47	Scheduled	Successfully assigned openshift-ingress/payload-processing-f9b8d9b9b-x5m47 to ip-10-0-142-151.ec2.internal
01:12:56	openshift-ingress		e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9	Scheduled	Successfully assigned openshift-ingress/e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9 to ip-10-0-142-236.ec2.internal
00:52:59	openshift-ingress		istiod-openshift-gateway-7574459656-7gk7f	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-7574459656-7gk7f to ip-10-0-142-236.ec2.internal
00:53:05	openshift-ingress		istiod-openshift-gateway-94bb8fbfd-8nk6j	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8nk6j to ip-10-0-138-126.ec2.internal
01:19:27	openshift-ingress		e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	Scheduled	Successfully assigned openshift-ingress/e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4 to ip-10-0-142-236.ec2.internal
01:19:19	openshift-ingress		e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Scheduled	Successfully assigned openshift-ingress/e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4 to ip-10-0-142-236.ec2.internal
00:52:58	openshift-ingress		kube-auth-proxy-5f9949ccdc-csscl	Scheduled	Successfully assigned openshift-ingress/kube-auth-proxy-5f9949ccdc-csscl to ip-10-0-142-236.ec2.internal
00:52:58	openshift-ingress		kube-auth-proxy-5f9949ccdc-vktkp	Scheduled	Successfully assigned openshift-ingress/kube-auth-proxy-5f9949ccdc-vktkp to ip-10-0-142-151.ec2.internal
01:12:31	openshift-ingress		e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x	Scheduled	Successfully assigned openshift-ingress/e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x to ip-10-0-142-236.ec2.internal
01:14:46	openshift-ingress		e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Scheduled	Successfully assigned openshift-ingress/e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn to ip-10-0-142-236.ec2.internal
00:48:30	openshift-ingress		router-default-5cb5fd56db-fx8qh	Scheduled	Successfully assigned openshift-ingress/router-default-5cb5fd56db-fx8qh to ip-10-0-142-236.ec2.internal
01:18:37	openshift-ingress		e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	Scheduled	Successfully assigned openshift-ingress/e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt to ip-10-0-142-236.ec2.internal
00:55:21	openshift-ingress		maas-default-gateway-openshift-default-56654dcf7-8hzn8	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-8hzn8 to ip-10-0-142-236.ec2.internal
01:16:30	openshift-ingress		e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	Scheduled	Successfully assigned openshift-ingress/e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s to ip-10-0-142-236.ec2.internal
00:56:32	openshift-ingress		payload-pre-processing-74584bb9f8-4qj5j	Scheduled	Successfully assigned openshift-ingress/payload-pre-processing-74584bb9f8-4qj5j to ip-10-0-142-151.ec2.internal
01:16:22	openshift-ingress		e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Scheduled	Successfully assigned openshift-ingress/e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr to ip-10-0-142-236.ec2.internal
00:53:05	openshift-ingress		data-science-gateway-data-science-gateway-class-6576859cf4m42gd	Scheduled	Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4m42gd to ip-10-0-142-236.ec2.internal
01:18:28	openshift-ingress		e2e-api-a-314e69-openshift-default-69675d5c87-677k5	Scheduled	Successfully assigned openshift-ingress/e2e-api-a-314e69-openshift-default-69675d5c87-677k5 to ip-10-0-142-236.ec2.internal
01:12:44	openshift-ingress		e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws	Scheduled	Successfully assigned openshift-ingress/e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws to ip-10-0-142-236.ec2.internal
00:38:09	openshift-ingress	deployment-controller	router-default	ScalingReplicaSet	Scaled up replica set router-default-5cb5fd56db from 0 to 1
00:38:09 (x7)	openshift-ingress	replicaset-controller	router-default-5cb5fd56db	FailedCreate	Error creating: pods "router-default-5cb5fd56db-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
00:38:43	openshift-ingress	service-controller	router-default	UnAvailableLoadBalancer	There are no available nodes for LoadBalancer
00:38:43	openshift-ingress	service-controller	router-default	EnsuringLoadBalancer	Ensuring load balancer
00:38:48	openshift-ingress	service-controller	router-default	EnsuredLoadBalancer	Ensured load balancer
00:43:57 (x17)	openshift-ingress	replicaset-controller	router-default-5cb5fd56db	FailedCreate	Error creating: pods "router-default-5cb5fd56db-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
00:44:24 (x3)	openshift-ingress	service-controller	router-default	UpdatedLoadBalancer	Updated load balancer with new hosts
00:48:30	openshift-ingress	replicaset-controller	router-default-5cb5fd56db	SuccessfulCreate	Created pod: router-default-5cb5fd56db-fx8qh
00:48:37 (x5)	openshift-ingress	kubelet	router-default-5cb5fd56db-fx8qh	FailedMount	MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
00:48:46 (x6)	openshift-ingress	kubelet	router-default-5cb5fd56db-fx8qh	FailedMount	MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
00:49:03	openshift-ingress	multus	router-default-5cb5fd56db-fx8qh	AddedInterface	Add eth0 [10.133.0.15/23] from ovn-kubernetes
00:49:03	openshift-ingress	kubelet	router-default-5cb5fd56db-fx8qh	Started	Started container router
00:49:03	openshift-ingress	kubelet	router-default-5cb5fd56db-fx8qh	Created	Created container: router
00:49:03	openshift-ingress	kubelet	router-default-5cb5fd56db-fx8qh	Pulled	Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:0595b1d6de8e08ddb61ce581075357331193db6206d341c15cf213514ef0f693" already present on machine
00:52:57	openshift-ingress	deployment-controller	kube-auth-proxy	ScalingReplicaSet	Scaled up replica set kube-auth-proxy-5f9949ccdc from 0 to 2
00:52:58	openshift-ingress	multus	kube-auth-proxy-5f9949ccdc-vktkp	AddedInterface	Add eth0 [10.132.0.24/23] from ovn-kubernetes
00:52:58	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-vktkp	Pulling	Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"
00:52:58	openshift-ingress	multus	kube-auth-proxy-5f9949ccdc-csscl	AddedInterface	Add eth0 [10.133.0.29/23] from ovn-kubernetes
00:52:58	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-csscl	Pulling	Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"
00:52:58 (x5)	openshift-ingress	replicaset-controller	kube-auth-proxy-5f9949ccdc	FailedCreate	Error creating: pods "kube-auth-proxy-5f9949ccdc-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found
00:52:58	openshift-ingress	replicaset-controller	kube-auth-proxy-5f9949ccdc	SuccessfulCreate	Created pod: kube-auth-proxy-5f9949ccdc-csscl
00:52:58	openshift-ingress	replicaset-controller	kube-auth-proxy-5f9949ccdc	SuccessfulCreate	Created pod: kube-auth-proxy-5f9949ccdc-vktkp
00:52:59	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7574459656	SuccessfulCreate	Created pod: istiod-openshift-gateway-7574459656-7gk7f
00:52:59	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-7574459656 from 0 to 1
00:53:00	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"
00:53:00	openshift-ingress	multus	istiod-openshift-gateway-7574459656-7gk7f	AddedInterface	Add eth0 [10.133.0.32/23] from ovn-kubernetes
00:53:01	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-vktkp	Pulled	Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.896s (2.896s including waiting). Image size: 265120132 bytes.
00:53:01	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-vktkp	Started	Started container kube-auth-proxy
00:53:01	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-vktkp	Created	Created container: kube-auth-proxy
00:53:03	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-csscl	Started	Started container kube-auth-proxy
00:53:03	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-csscl	Pulled	Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 4.67s (4.67s including waiting). Image size: 265120132 bytes.
00:53:03	openshift-ingress	kubelet	kube-auth-proxy-5f9949ccdc-csscl	Created	Created container: kube-auth-proxy
00:53:04	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	Started	Started container discovery
00:53:04	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 4.356s (4.356s including waiting). Image size: 181034515 bytes.
00:53:04	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	Created	Created container: discovery
00:53:05	openshift-ingress	deployment-controller	data-science-gateway-data-science-gateway-class	ScalingReplicaSet	Scaled up replica set data-science-gateway-data-science-gateway-class-6576859cf4 from 0 to 1
00:53:05	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	Unhealthy	Readiness probe failed: HTTP probe failed with statuscode: 503
00:53:05	openshift-ingress	replicaset-controller	data-science-gateway-data-science-gateway-class-6576859cf4	SuccessfulCreate	Created pod: data-science-gateway-data-science-gateway-class-6576859cf4m42gd
00:53:05	openshift-ingress	replicaset-controller	istiod-openshift-gateway-94bb8fbfd	SuccessfulCreate	Created pod: istiod-openshift-gateway-94bb8fbfd-8nk6j
00:53:05	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	ProbeError	Readiness probe error: HTTP probe failed with statuscode: 503 body:
00:53:05	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1
00:53:06	openshift-ingress	multus	data-science-gateway-data-science-gateway-class-6576859cf4m42gd	AddedInterface	Add eth0 [10.133.0.33/23] from ovn-kubernetes
00:53:06	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-8nk6j	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"
00:53:06	openshift-ingress	multus	istiod-openshift-gateway-94bb8fbfd-8nk6j	AddedInterface	Add eth0 [10.134.0.21/23] from ovn-kubernetes
00:53:06	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4m42gd	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"
00:53:08	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-8nk6j	Started	Started container discovery
00:53:08	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-8nk6j	Created	Created container: discovery
00:53:08	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-8nk6j	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 2.205s (2.205s including waiting). Image size: 181034515 bytes.
00:53:09	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4m42gd	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 2.708s (2.708s including waiting). Image size: 181251556 bytes.
00:53:09	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4m42gd	Started	Started container istio-proxy
00:53:09	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4m42gd	Created	Created container: istio-proxy
00:53:09	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-8nk6j	ProbeError	Readiness probe error: HTTP probe failed with statuscode: 503 body:
00:53:09	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-8nk6j	Unhealthy	Readiness probe failed: HTTP probe failed with statuscode: 503
00:53:10	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled down replica set istiod-openshift-gateway-7574459656 from 1 to 0
00:53:10	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-7gk7f	Killing	Stopping container discovery
00:53:10	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7574459656	SuccessfulDelete	Deleted pod: istiod-openshift-gateway-7574459656-7gk7f
00:53:13	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
00:53:13	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
00:53:28	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
00:53:28	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
00:53:44 (x3)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
00:53:44 (x3)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
00:55:21	openshift-ingress	deployment-controller	maas-default-gateway-openshift-default	ScalingReplicaSet	Scaled up replica set maas-default-gateway-openshift-default-56654dcf7 from 0 to 1
00:55:21	openshift-ingress	gateway_labeler_controller	maas-default-gateway	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway
00:55:21	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-56654dcf7	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-56654dcf7-8hzn8
00:55:22	openshift-ingress	multus	maas-default-gateway-openshift-default-56654dcf7-8hzn8	AddedInterface	Add eth0 [10.133.0.49/23] from ovn-kubernetes
00:56:32 (x5)	openshift-ingress	replicaset-controller	payload-processing-f9b8d9b9b	FailedCreate	Error creating: pods "payload-processing-f9b8d9b9b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
00:56:32	openshift-ingress	kubelet	payload-pre-processing-74584bb9f8-4qj5j	Pulling	Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0"
00:56:32	openshift-ingress	multus	payload-pre-processing-74584bb9f8-4qj5j	AddedInterface	Add eth0 [10.132.0.34/23] from ovn-kubernetes
00:56:32	openshift-ingress	replicaset-controller	payload-processing-f9b8d9b9b	SuccessfulCreate	Created pod: payload-processing-f9b8d9b9b-x5m47
00:56:32	openshift-ingress	kubelet	payload-processing-f9b8d9b9b-x5m47	Pulling	Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0"
00:56:32	openshift-ingress	multus	payload-processing-f9b8d9b9b-x5m47	AddedInterface	Add eth0 [10.132.0.35/23] from ovn-kubernetes
00:56:32	openshift-ingress	deployment-controller	payload-pre-processing	ScalingReplicaSet	Scaled up replica set payload-pre-processing-74584bb9f8 from 0 to 1
00:56:32	openshift-ingress	replicaset-controller	payload-pre-processing-74584bb9f8	SuccessfulCreate	Created pod: payload-pre-processing-74584bb9f8-4qj5j
00:56:32 (x4)	openshift-ingress	replicaset-controller	payload-pre-processing-74584bb9f8	FailedCreate	Error creating: pods "payload-pre-processing-74584bb9f8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
00:56:32	openshift-ingress	deployment-controller	payload-processing	ScalingReplicaSet	Scaled up replica set payload-processing-f9b8d9b9b from 0 to 1
00:56:34	openshift-ingress	kubelet	payload-pre-processing-74584bb9f8-4qj5j	Started	Started container payload-pre-processing
00:56:34	openshift-ingress	kubelet	payload-pre-processing-74584bb9f8-4qj5j	Pulled	Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0" in 1.477s (1.477s including waiting). Image size: 165432098 bytes.
00:56:34	openshift-ingress	kubelet	payload-pre-processing-74584bb9f8-4qj5j	Created	Created container: payload-pre-processing
00:56:34	openshift-ingress	kubelet	payload-processing-f9b8d9b9b-x5m47	Pulled	Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0" in 1.439s (1.439s including waiting). Image size: 165432098 bytes.
00:56:34	openshift-ingress	kubelet	payload-processing-f9b8d9b9b-x5m47	Created	Created container: payload-processing
00:56:34	openshift-ingress	kubelet	payload-processing-f9b8d9b9b-x5m47	Started	Started container payload-processing
00:56:39	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	ProbeError	Readiness probe error: Get "http://10.133.0.49:15021/healthz/ready": read tcp 10.133.0.2:57882->10.133.0.49:15021: read: connection reset by peer body:
00:56:39	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	Unhealthy	Readiness probe failed: Get "http://10.133.0.49:15021/healthz/ready": read tcp 10.133.0.2:57882->10.133.0.49:15021: read: connection reset by peer
00:56:40 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
00:56:40 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	Created	Created container: istio-proxy
00:56:40 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	Started	Started container istio-proxy
00:56:44 (x4)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	ProbeError	Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused body:
00:56:44 (x4)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-8hzn8	Unhealthy	Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused
01:12:30	openshift-ingress	deployment-controller	e2e-ait-f9c9a9ad-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-ait-f9c9a9ad-openshift-default-d9c59c95f from 0 to 1
01:12:30	openshift-ingress	gateway_labeler_controller	e2e-ait-f9c9a9ad	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f9c9a9ad
01:12:31	openshift-ingress	kubelet	e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x	Started	Started container istio-proxy
01:12:31	openshift-ingress	replicaset-controller	e2e-ait-f9c9a9ad-openshift-default-d9c59c95f	SuccessfulCreate	Created pod: e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x
01:12:31	openshift-ingress	service-controller	e2e-ait-f9c9a9ad-openshift-default	EnsuringLoadBalancer	Ensuring load balancer
01:12:31	openshift-ingress	kubelet	e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x	Created	Created container: istio-proxy
01:12:31	openshift-ingress	kubelet	e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:12:31	openshift-ingress	multus	e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x	AddedInterface	Add eth0 [10.133.0.57/23] from ovn-kubernetes
01:12:35	openshift-ingress	service-controller	e2e-ait-f9c9a9ad-openshift-default	EnsuredLoadBalancer	Ensured load balancer
01:12:37	openshift-ingress	kubelet	e2e-ait-f9c9a9ad-openshift-default-d9c59c95f-xhp9x	Killing	Stopping container istio-proxy
01:12:37	openshift-ingress	service-controller	e2e-ait-f9c9a9ad-openshift-default	DeletingLoadBalancer	Deleting load balancer
01:12:39	openshift-ingress	service-controller	e2e-ait-f9c9a9ad-openshift-default	DeletedLoadBalancer	Deleted load balancer
01:12:44	openshift-ingress	deployment-controller	e2e-migrate-389a944b-gw-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc from 0 to 1
01:12:44	openshift-ingress	service-controller	e2e-migrate-389a944b-gw-openshift-default	EnsuringLoadBalancer	Ensuring load balancer
01:12:44	openshift-ingress	gateway_labeler_controller	e2e-migrate-389a944b-gw	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-389a944b-gw
01:12:44	openshift-ingress	replicaset-controller	e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc	SuccessfulCreate	Created pod: e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws
01:12:44	openshift-ingress	kubelet	e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws	Started	Started container istio-proxy
01:12:44	openshift-ingress	kubelet	e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws	Created	Created container: istio-proxy
01:12:44	openshift-ingress	multus	e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws	AddedInterface	Add eth0 [10.133.0.58/23] from ovn-kubernetes
01:12:44	openshift-ingress	kubelet	e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:12:47	openshift-ingress	service-controller	e2e-migrate-389a944b-gw-openshift-default	EnsuredLoadBalancer	Ensured load balancer
01:12:50	openshift-ingress	service-controller	e2e-migrate-389a944b-gw-openshift-default	DeletingLoadBalancer	Deleting load balancer
01:12:50	openshift-ingress	kubelet	e2e-migrate-389a944b-gw-openshift-default-7fcd485dbc-mvwws	Killing	Stopping container istio-proxy
01:12:53	openshift-ingress	service-controller	e2e-migrate-389a944b-gw-openshift-default	DeletedLoadBalancer	Deleted load balancer
01:12:56	openshift-ingress	gateway_labeler_controller	e2e-derive-f122a48d	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-derive-f122a48d
01:12:56	openshift-ingress	service-controller	e2e-derive-f122a48d-openshift-default	EnsuringLoadBalancer	Ensuring load balancer
01:12:56	openshift-ingress	multus	e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9	AddedInterface	Add eth0 [10.133.0.59/23] from ovn-kubernetes
01:12:56	openshift-ingress	kubelet	e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:12:56	openshift-ingress	kubelet	e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9	Created	Created container: istio-proxy
01:12:56	openshift-ingress	kubelet	e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9	Started	Started container istio-proxy
01:12:56	openshift-ingress	replicaset-controller	e2e-derive-f122a48d-openshift-default-554485d5f5	SuccessfulCreate	Created pod: e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9
01:12:56	openshift-ingress	deployment-controller	e2e-derive-f122a48d-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-derive-f122a48d-openshift-default-554485d5f5 from 0 to 1
01:13:00	openshift-ingress	service-controller	e2e-derive-f122a48d-openshift-default	EnsuredLoadBalancer	Ensured load balancer
01:13:02	openshift-ingress	service-controller	e2e-derive-f122a48d-openshift-default	DeletingLoadBalancer	Deleting load balancer
01:13:02	openshift-ingress	kubelet	e2e-derive-f122a48d-openshift-default-554485d5f5-88kb9	Killing	Stopping container istio-proxy
01:13:04	openshift-ingress	service-controller	e2e-derive-f122a48d-openshift-default	DeletedLoadBalancer	Deleted load balancer
01:14:45	openshift-ingress	gateway_labeler_controller	e2e-mt-54ada037	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-mt-54ada037
01:14:45	openshift-ingress	deployment-controller	e2e-mt-54ada037-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-mt-54ada037-openshift-default-ccdf877c5 from 0 to 1
01:14:46	openshift-ingress	multus	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	AddedInterface	Add eth0 [10.133.0.60/23] from ovn-kubernetes
01:14:46	openshift-ingress	replicaset-controller	e2e-mt-54ada037-openshift-default-ccdf877c5	SuccessfulCreate	Created pod: e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn
01:15:03 (x2)	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:15:03 (x2)	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Created	Created container: istio-proxy
01:15:03	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	ProbeError	Readiness probe error: Get "http://10.133.0.60:15021/healthz/ready": read tcp 10.133.0.2:59152->10.133.0.60:15021: read: connection reset by peer body:
01:15:03	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Unhealthy	Readiness probe failed: Get "http://10.133.0.60:15021/healthz/ready": read tcp 10.133.0.2:59152->10.133.0.60:15021: read: connection reset by peer
01:15:04 (x2)	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Started	Started container istio-proxy
01:15:08 (x4)	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	ProbeError	Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused body:
01:15:08 (x4)	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Unhealthy	Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused
01:15:08	openshift-ingress	kubelet	e2e-mt-54ada037-openshift-default-ccdf877c5-hd7nn	Killing	Stopping container istio-proxy
01:16:22	openshift-ingress	replicaset-controller	e2e-inf-a-53347c-openshift-default-847599ccdc	SuccessfulCreate	Created pod: e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr
01:16:22	openshift-ingress	gateway_labeler_controller	e2e-inf-a-53347c	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-53347c
01:16:22	openshift-ingress	deployment-controller	e2e-inf-a-53347c-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-inf-a-53347c-openshift-default-847599ccdc from 0 to 1
01:16:23	openshift-ingress	multus	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	AddedInterface	Add eth0 [10.133.0.61/23] from ovn-kubernetes
01:16:30	openshift-ingress	multus	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	AddedInterface	Add eth0 [10.133.0.62/23] from ovn-kubernetes
01:16:30	openshift-ingress	gateway_labeler_controller	e2e-inf-b-640a35	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-640a35
01:16:30	openshift-ingress	deployment-controller	e2e-inf-b-640a35-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-inf-b-640a35-openshift-default-7b7bfd9d7c from 0 to 1
01:16:30	openshift-ingress	replicaset-controller	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c	SuccessfulCreate	Created pod: e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s
01:16:40	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Unhealthy	Readiness probe failed: Get "http://10.133.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
01:16:40	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	ProbeError	Readiness probe error: Get "http://10.133.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
01:16:42 (x2)	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Started	Started container istio-proxy
01:16:42 (x2)	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Created	Created container: istio-proxy
01:16:42 (x2)	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:16:47 (x4)	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	ProbeError	Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:
01:16:47 (x4)	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Unhealthy	Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused
01:16:51 (x2)	openshift-ingress	kubelet	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	Created	Created container: istio-proxy
01:16:51 (x2)	openshift-ingress	kubelet	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:16:51 (x2)	openshift-ingress	kubelet	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	Started	Started container istio-proxy
01:16:55 (x4)	openshift-ingress	kubelet	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	ProbeError	Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:
01:16:55 (x4)	openshift-ingress	kubelet	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	Unhealthy	Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused
01:18:20	openshift-ingress	kubelet	e2e-inf-a-53347c-openshift-default-847599ccdc-gvwdr	Killing	Stopping container istio-proxy
01:18:27	openshift-ingress	kubelet	e2e-inf-b-640a35-openshift-default-7b7bfd9d7c-f4h7s	Killing	Stopping container istio-proxy
01:18:28	openshift-ingress	deployment-controller	e2e-api-a-314e69-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-api-a-314e69-openshift-default-69675d5c87 from 0 to 1
01:18:28	openshift-ingress	replicaset-controller	e2e-api-a-314e69-openshift-default-69675d5c87	SuccessfulCreate	Created pod: e2e-api-a-314e69-openshift-default-69675d5c87-677k5
01:18:28	openshift-ingress	gateway_labeler_controller	e2e-api-a-314e69	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-314e69
01:18:29	openshift-ingress	multus	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	AddedInterface	Add eth0 [10.133.0.64/23] from ovn-kubernetes
01:18:37	openshift-ingress	gateway_labeler_controller	e2e-api-b-edba78	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-edba78
01:18:37	openshift-ingress	multus	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	AddedInterface	Add eth0 [10.133.0.65/23] from ovn-kubernetes
01:18:37	openshift-ingress	replicaset-controller	e2e-api-b-edba78-openshift-default-8c8865bf9	SuccessfulCreate	Created pod: e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt
01:18:37	openshift-ingress	deployment-controller	e2e-api-b-edba78-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-api-b-edba78-openshift-default-8c8865bf9 from 0 to 1
01:18:44 (x2)	openshift-ingress	kubelet	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:18:44 (x2)	openshift-ingress	kubelet	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	Created	Created container: istio-proxy
01:18:44 (x2)	openshift-ingress	kubelet	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	Started	Started container istio-proxy
01:18:49 (x4)	openshift-ingress	kubelet	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	Unhealthy	Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused
01:18:49 (x4)	openshift-ingress	kubelet	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	ProbeError	Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused body:
01:18:52 (x2)	openshift-ingress	kubelet	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	Started	Started container istio-proxy
01:18:52 (x2)	openshift-ingress	kubelet	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	Created	Created container: istio-proxy
01:18:52 (x2)	openshift-ingress	kubelet	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:18:56 (x4)	openshift-ingress	kubelet	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	Unhealthy	Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused
01:18:56 (x4)	openshift-ingress	kubelet	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	ProbeError	Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
01:19:05	openshift-ingress	kubelet	e2e-api-a-314e69-openshift-default-69675d5c87-677k5	Killing	Stopping container istio-proxy
01:19:18	openshift-ingress	gateway_labeler_controller	e2e-shared-a-44361b	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-44361b
01:19:18	openshift-ingress	kubelet	e2e-api-b-edba78-openshift-default-8c8865bf9-cftnt	Killing	Stopping container istio-proxy
01:19:19	openshift-ingress	deployment-controller	e2e-shared-a-44361b-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-shared-a-44361b-openshift-default-6485bcb99f from 0 to 1
01:19:19	openshift-ingress	replicaset-controller	e2e-shared-a-44361b-openshift-default-6485bcb99f	SuccessfulCreate	Created pod: e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4
01:19:19	openshift-ingress	multus	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	AddedInterface	Add eth0 [10.133.0.66/23] from ovn-kubernetes
01:19:27	openshift-ingress	deployment-controller	e2e-shared-b-908f0c-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-shared-b-908f0c-openshift-default-c9d78576c from 0 to 1
01:19:27	openshift-ingress	replicaset-controller	e2e-shared-b-908f0c-openshift-default-c9d78576c	SuccessfulCreate	Created pod: e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4
01:19:27	openshift-ingress	multus	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	AddedInterface	Add eth0 [10.133.0.67/23] from ovn-kubernetes
01:19:27	openshift-ingress	gateway_labeler_controller	e2e-shared-b-908f0c	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-908f0c
01:19:39 (x2)	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Started	Started container istio-proxy
01:19:39 (x2)	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Created	Created container: istio-proxy
01:19:39 (x2)	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:19:44 (x4)	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Unhealthy	Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
01:19:44 (x4)	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	ProbeError	Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:
01:19:49 (x2)	openshift-ingress	kubelet	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	Started	Started container istio-proxy
01:19:49 (x2)	openshift-ingress	kubelet	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	Created	Created container: istio-proxy
01:19:49 (x2)	openshift-ingress	kubelet	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
01:19:54 (x5)	openshift-ingress	kubelet	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	Unhealthy	Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused
01:19:54 (x5)	openshift-ingress	kubelet	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	ProbeError	Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:
01:26:26	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Killing	Stopping container istio-proxy
01:26:30	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	Unhealthy	Readiness probe failed: HTTP probe failed with statuscode: 503
01:26:30	openshift-ingress	kubelet	e2e-shared-a-44361b-openshift-default-6485bcb99f-vn7c4	ProbeError	Readiness probe error: HTTP probe failed with statuscode: 503 body:
01:26:33	openshift-ingress	kubelet	e2e-shared-b-908f0c-openshift-default-c9d78576c-d6jt4	Killing	Stopping container istio-proxy