Events

Time	Namespace	Component	RelatedObject	Reason	Message
18:33:23	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-pjmlx	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-pjmlx to ip-10-0-130-63.us-east-2.compute.internal
18:32:37	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-dfn2b	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-dfn2b to ip-10-0-130-59.us-east-2.compute.internal
18:32:37	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-vlld6	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-vlld6 to ip-10-0-130-63.us-east-2.compute.internal
18:33:22	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-x9znl	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-x9znl to ip-10-0-130-59.us-east-2.compute.internal
18:25:47	openshift-ingress		istiod-openshift-gateway-7876bfcb74-xdc7v	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-xdc7v to ip-10-0-130-59.us-east-2.compute.internal
18:27:30	openshift-ingress		kube-auth-proxy-7dcb6fdfc4-c4f4k	Scheduled	Successfully assigned openshift-ingress/kube-auth-proxy-7dcb6fdfc4-c4f4k to ip-10-0-130-59.us-east-2.compute.internal
18:25:52	openshift-ingress		openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg	Scheduled	Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg to ip-10-0-130-63.us-east-2.compute.internal
18:25:52	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-skbq4	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-skbq4 to ip-10-0-130-63.us-east-2.compute.internal
18:28:26	openshift-ingress		istiod-openshift-gateway-7ddd9cfdc8-5fc9b	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-5fc9b to ip-10-0-130-59.us-east-2.compute.internal
18:35:12	openshift-ingress		e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Scheduled	Successfully assigned openshift-ingress/e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd to ip-10-0-130-63.us-east-2.compute.internal
18:33:22	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-r9hdg	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-r9hdg to ip-10-0-143-124.us-east-2.compute.internal
18:32:37	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-vdw7q	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-vdw7q to ip-10-0-143-124.us-east-2.compute.internal
18:32:52	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-fnj8g	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-fnj8g to ip-10-0-130-59.us-east-2.compute.internal
18:35:16	openshift-ingress		payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d	Scheduled	Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d to ip-10-0-130-63.us-east-2.compute.internal
18:31:34	openshift-ingress		payload-processing-55fbbbf9dc-hktvc	Scheduled	Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-hktvc to ip-10-0-130-63.us-east-2.compute.internal
18:13:55	openshift-ingress		router-default-bc466ddff-fq6ch	FailedScheduling	no nodes available to schedule pods
18:13:53	openshift-ingress		router-default-bc466ddff-fq6ch	FailedScheduling	no nodes available to schedule pods
18:35:36	openshift-ingress		payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	Scheduled	Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5 to ip-10-0-130-63.us-east-2.compute.internal
18:33:22	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-6mddl	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-6mddl to ip-10-0-130-63.us-east-2.compute.internal
18:20:19	openshift-ingress		router-default-bc466ddff-fq6ch	Scheduled	Successfully assigned openshift-ingress/router-default-bc466ddff-fq6ch to ip-10-0-130-59.us-east-2.compute.internal
18:32:52	openshift-ingress		maas-default-gateway-openshift-default-55fbd65575-5cxxt	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55fbd65575-5cxxt to ip-10-0-143-124.us-east-2.compute.internal
18:35:16	openshift-ingress		payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf	Scheduled	Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf to ip-10-0-130-63.us-east-2.compute.internal
18:31:34	openshift-ingress		payload-pre-processing-7b98bb678c-csx9p	Scheduled	Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-csx9p to ip-10-0-130-63.us-east-2.compute.internal
18:27:30	openshift-ingress		data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl	Scheduled	Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl to ip-10-0-130-63.us-east-2.compute.internal
18:27:30	openshift-ingress		kube-auth-proxy-7dcb6fdfc4-qp4c9	Scheduled	Successfully assigned openshift-ingress/kube-auth-proxy-7dcb6fdfc4-qp4c9 to ip-10-0-143-124.us-east-2.compute.internal
18:35:35	openshift-ingress		payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	Scheduled	Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq to ip-10-0-130-63.us-east-2.compute.internal
18:35:30	openshift-ingress		e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Scheduled	Successfully assigned openshift-ingress/e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr to ip-10-0-143-124.us-east-2.compute.internal
18:13:25	openshift-ingress	deployment-controller	router-default	ScalingReplicaSet	Scaled up replica set router-default-bc466ddff from 0 to 1
18:13:25	openshift-ingress	service-controller	router-default	UnAvailableLoadBalancer	There are no available nodes for LoadBalancer
18:13:25	openshift-ingress	service-controller	router-default	EnsuringLoadBalancer	Ensuring load balancer
18:13:28	openshift-ingress	service-controller	router-default	EnsuredLoadBalancer	Ensured load balancer
18:13:53	openshift-ingress	replicaset-controller	router-default-bc466ddff	SuccessfulCreate	Created pod: router-default-bc466ddff-fq6ch
18:19:59 (x3)	openshift-ingress	service-controller	router-default	UpdatedLoadBalancer	Updated load balancer with new hosts
18:20:27 (x5)	openshift-ingress	kubelet	router-default-bc466ddff-fq6ch	FailedMount	MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
18:20:27 (x5)	openshift-ingress	kubelet	router-default-bc466ddff-fq6ch	FailedMount	MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
18:20:32	openshift-ingress	multus	router-default-bc466ddff-fq6ch	AddedInterface	Add eth0 [10.132.0.9/23] from ovn-kubernetes
18:20:32	openshift-ingress	kubelet	router-default-bc466ddff-fq6ch	Pulled	Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod
18:20:32	openshift-ingress	kubelet	router-default-bc466ddff-fq6ch	Created	Container created
18:20:32	openshift-ingress	kubelet	router-default-bc466ddff-fq6ch	Started	Container started
18:25:45	openshift-ingress	gateway_labeler_controller	maas-default-gateway	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway
18:25:47	openshift-ingress	kubelet	istiod-openshift-gateway-7876bfcb74-xdc7v	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"
18:25:47	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1
18:25:47	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7876bfcb74	SuccessfulCreate	Created pod: istiod-openshift-gateway-7876bfcb74-xdc7v
18:25:47	openshift-ingress	multus	istiod-openshift-gateway-7876bfcb74-xdc7v	AddedInterface	Add eth0 [10.132.0.26/23] from ovn-kubernetes
18:25:51	openshift-ingress	kubelet	istiod-openshift-gateway-7876bfcb74-xdc7v	Created	Container created
18:25:51	openshift-ingress	kubelet	istiod-openshift-gateway-7876bfcb74-xdc7v	Started	Container started
18:25:51	openshift-ingress	kubelet	istiod-openshift-gateway-7876bfcb74-xdc7v	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.679s (3.679s including waiting). Image size: 185476477 bytes.
18:25:52	openshift-ingress	replicaset-controller	openshift-ai-inference-openshift-ai-inference-645b5cfbff	SuccessfulCreate	Created pod: openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg
18:25:52	openshift-ingress	deployment-controller	openshift-ai-inference-openshift-ai-inference	ScalingReplicaSet	Scaled up replica set openshift-ai-inference-openshift-ai-inference-645b5cfbff from 0 to 1
18:25:52	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-skbq4	AddedInterface	Add eth0 [10.134.0.28/23] from ovn-kubernetes
18:25:52	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-skbq4
18:25:52	openshift-ingress	deployment-controller	maas-default-gateway-openshift-default	ScalingReplicaSet	Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 0 to 1
18:25:52	openshift-ingress	service-controller	openshift-ai-inference-openshift-ai-inference	EnsuringLoadBalancer	Ensuring load balancer
18:25:53	openshift-ingress	multus	openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg	AddedInterface	Add eth0 [10.134.0.29/23] from ovn-kubernetes
18:25:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"
18:25:53	openshift-ingress	kubelet	openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"
18:25:55	openshift-ingress	service-controller	openshift-ai-inference-openshift-ai-inference	EnsuredLoadBalancer	Ensured load balancer
18:26:02	openshift-ingress	kubelet	openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg	Created	Container created
18:26:02	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.598s (9.598s including waiting). Image size: 183555333 bytes.
18:26:02	openshift-ingress	kubelet	openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 9.79s (9.79s including waiting). Image size: 183555333 bytes.
18:26:03	openshift-ingress	kubelet	openshift-ai-inference-openshift-ai-inference-645b5cfbff-crwlg	Started	Container started
18:26:37 (x3)	openshift-ingress	horizontal-pod-autoscaler	openshift-ai-inference-openshift-ai-inference	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:26:37 (x3)	openshift-ingress	horizontal-pod-autoscaler	openshift-ai-inference-openshift-ai-inference	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:27:30	openshift-ingress	replicaset-controller	data-science-gateway-data-science-gateway-class-bb66448f7	SuccessfulCreate	Created pod: data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl
18:27:30	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl	Created	Container created
18:27:30	openshift-ingress	deployment-controller	data-science-gateway-data-science-gateway-class	ScalingReplicaSet	Scaled up replica set data-science-gateway-data-science-gateway-class-bb66448f7 from 0 to 1
18:27:30	openshift-ingress	multus	kube-auth-proxy-7dcb6fdfc4-c4f4k	AddedInterface	Add eth0 [10.132.0.28/23] from ovn-kubernetes
18:27:30	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-c4f4k	Pulling	Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"
18:27:30	openshift-ingress	replicaset-controller	kube-auth-proxy-7dcb6fdfc4	SuccessfulCreate	Created pod: kube-auth-proxy-7dcb6fdfc4-qp4c9
18:27:30	openshift-ingress	multus	data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl	AddedInterface	Add eth0 [10.134.0.39/23] from ovn-kubernetes
18:27:30	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl	Started	Container started
18:27:30	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-bb66448f7-lkqcl	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:27:30	openshift-ingress	multus	kube-auth-proxy-7dcb6fdfc4-qp4c9	AddedInterface	Add eth0 [10.133.0.24/23] from ovn-kubernetes
18:27:30	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-qp4c9	Pulling	Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"
18:27:30	openshift-ingress	deployment-controller	kube-auth-proxy	ScalingReplicaSet	Scaled up replica set kube-auth-proxy-7dcb6fdfc4 from 0 to 2
18:27:30	openshift-ingress	replicaset-controller	kube-auth-proxy-7dcb6fdfc4	SuccessfulCreate	Created pod: kube-auth-proxy-7dcb6fdfc4-c4f4k
18:27:33	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-qp4c9	Pulled	Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.427s (2.427s including waiting). Image size: 188904677 bytes.
18:27:33	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-qp4c9	Started	Container started
18:27:33	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-qp4c9	Created	Container created
18:27:33	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-c4f4k	Created	Container created
18:27:33	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-c4f4k	Pulled	Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.48s (2.48s including waiting). Image size: 188904677 bytes.
18:27:33	openshift-ingress	kubelet	kube-auth-proxy-7dcb6fdfc4-c4f4k	Started	Container started
18:27:47	openshift-ingress	horizontal-pod-autoscaler	data-science-gateway-data-science-gateway-class	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:27:47	openshift-ingress	horizontal-pod-autoscaler	data-science-gateway-data-science-gateway-class	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:28:00 (x2)	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:28:00 (x2)	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:28:15	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:28:15	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:28:26	openshift-ingress	kubelet	istiod-openshift-gateway-7ddd9cfdc8-5fc9b	Created	Container created
18:28:26	openshift-ingress	kubelet	istiod-openshift-gateway-7ddd9cfdc8-5fc9b	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod
18:28:26	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1
18:28:26	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7ddd9cfdc8	SuccessfulCreate	Created pod: istiod-openshift-gateway-7ddd9cfdc8-5fc9b
18:28:26	openshift-ingress	kubelet	istiod-openshift-gateway-7ddd9cfdc8-5fc9b	Started	Container started
18:28:26	openshift-ingress	multus	istiod-openshift-gateway-7ddd9cfdc8-5fc9b	AddedInterface	Add eth0 [10.132.0.34/23] from ovn-kubernetes
18:28:27	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0
18:28:27	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7876bfcb74	SuccessfulDelete	Deleted pod: istiod-openshift-gateway-7876bfcb74-xdc7v
18:28:27	openshift-ingress	kubelet	istiod-openshift-gateway-7876bfcb74-xdc7v	Killing	Stopping container discovery
18:28:32 (x3)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:28:32 (x3)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:28:47	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:28:47	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:29:34	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:29:34 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Started	Container started
18:29:34 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Created	Container created
18:29:37	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:29:37	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:29:39 (x5)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Unhealthy	Startup probe failed: Get "http://10.134.0.28:15021/healthz/ready": dial tcp 10.134.0.28:15021: connect: connection refused
18:29:39 (x5)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	ProbeError	Startup probe error: Get "http://10.134.0.28:15021/healthz/ready": dial tcp 10.134.0.28:15021: connect: connection refused body:
18:30:07 (x5)	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:30:07 (x5)	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:31:34	openshift-ingress	replicaset-controller	payload-pre-processing-7b98bb678c	SuccessfulCreate	Created pod: payload-pre-processing-7b98bb678c-csx9p
18:31:34 (x4)	openshift-ingress	replicaset-controller	payload-pre-processing-7b98bb678c	FailedCreate	Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
18:31:34	openshift-ingress	deployment-controller	payload-processing	ScalingReplicaSet	Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1
18:31:34	openshift-ingress	replicaset-controller	payload-processing-55fbbbf9dc	SuccessfulCreate	Created pod: payload-processing-55fbbbf9dc-hktvc
18:31:34	openshift-ingress	deployment-controller	payload-pre-processing	ScalingReplicaSet	Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1
18:31:34 (x5)	openshift-ingress	replicaset-controller	payload-processing-55fbbbf9dc	FailedCreate	Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
18:31:35	openshift-ingress	kubelet	payload-processing-55fbbbf9dc-hktvc	Pulling	Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"
18:31:35	openshift-ingress	multus	payload-processing-55fbbbf9dc-hktvc	AddedInterface	Add eth0 [10.134.0.47/23] from ovn-kubernetes
18:31:35	openshift-ingress	kubelet	payload-pre-processing-7b98bb678c-csx9p	Pulling	Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"
18:31:35	openshift-ingress	multus	payload-pre-processing-7b98bb678c-csx9p	AddedInterface	Add eth0 [10.134.0.46/23] from ovn-kubernetes
18:31:37	openshift-ingress	kubelet	payload-processing-55fbbbf9dc-hktvc	Pulled	Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.263s (2.263s including waiting). Image size: 125353963 bytes.
18:31:37	openshift-ingress	kubelet	payload-pre-processing-7b98bb678c-csx9p	Started	Container started
18:31:37	openshift-ingress	kubelet	payload-pre-processing-7b98bb678c-csx9p	Created	Container created
18:31:37	openshift-ingress	kubelet	payload-processing-55fbbbf9dc-hktvc	Created	Container created
18:31:37	openshift-ingress	kubelet	payload-pre-processing-7b98bb678c-csx9p	Pulled	Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.37s (2.37s including waiting). Image size: 125353963 bytes.
18:31:37	openshift-ingress	kubelet	payload-processing-55fbbbf9dc-hktvc	Started	Container started
18:32:37	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-dfn2b
18:32:37	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-vlld6
18:32:37	openshift-ingress	deployment-controller	maas-default-gateway-openshift-default	ScalingReplicaSet	Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 1 to 4
18:32:37	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-vdw7q
18:32:37	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	SuccessfulRescale	New size: 4; reason: cpu resource utilization (percentage of request) above target
18:32:38	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:32:38	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-vlld6	AddedInterface	Add eth0 [10.134.0.53/23] from ovn-kubernetes
18:32:38	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"
18:32:38	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"
18:32:38	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-vdw7q	AddedInterface	Add eth0 [10.133.0.38/23] from ovn-kubernetes
18:32:38	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	Started	Container started
18:32:38	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	Created	Container created
18:32:38	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-dfn2b	AddedInterface	Add eth0 [10.132.0.43/23] from ovn-kubernetes
18:32:41	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	Created	Container created
18:32:41	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.403s (3.403s including waiting). Image size: 183555333 bytes.
18:32:42	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	Started	Container started
18:32:42	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	Created	Container created
18:32:42	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.474s (3.474s including waiting). Image size: 183555333 bytes.
18:32:42	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	Started	Container started
18:32:45 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	Unhealthy	Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused
18:32:45 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	ProbeError	Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:
18:32:49 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	Unhealthy	Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused
18:32:49 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	ProbeError	Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused body:
18:32:49 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	ProbeError	Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused body:
18:32:49 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	Unhealthy	Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused
18:32:52	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	SuccessfulRescale	New size: 6; reason:
18:32:52	openshift-ingress	deployment-controller	maas-default-gateway-openshift-default	ScalingReplicaSet	Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 4 to 6
18:32:52	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-fnj8g
18:32:52	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-5cxxt
18:32:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	Started	Container started
18:32:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-5cxxt	Started	Container started
18:32:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-5cxxt	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:32:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:32:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	Created	Container created
18:32:53	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-fnj8g	AddedInterface	Add eth0 [10.132.0.44/23] from ovn-kubernetes
18:32:53	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-5cxxt	AddedInterface	Add eth0 [10.133.0.39/23] from ovn-kubernetes
18:32:53	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-5cxxt	Created	Container created
18:33:00 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-5cxxt	ProbeError	Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:
18:33:00 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	Unhealthy	Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused
18:33:00 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	ProbeError	Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused body:
18:33:00 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-5cxxt	Unhealthy	Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused
18:33:22	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-6mddl
18:33:22	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-x9znl
18:33:22	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-55fbd65575-r9hdg
18:33:22	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-55fbd65575	SuccessfulCreate	(combined from similar events): Created pod: maas-default-gateway-openshift-default-55fbd65575-pjmlx
18:33:22	openshift-ingress	horizontal-pod-autoscaler	maas-default-gateway-openshift-default	SuccessfulRescale	New size: 10; reason: cpu resource utilization (percentage of request) above target
18:33:22	openshift-ingress	deployment-controller	maas-default-gateway-openshift-default	ScalingReplicaSet	Scaled up replica set maas-default-gateway-openshift-default-55fbd65575 from 6 to 10
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	Started	Container started
18:33:23	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-r9hdg	AddedInterface	Add eth0 [10.133.0.41/23] from ovn-kubernetes
18:33:23	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-6mddl	AddedInterface	Add eth0 [10.134.0.56/23] from ovn-kubernetes
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	Created	Container created
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	Started	Container started
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	Created	Container created
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:33:23	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-x9znl	AddedInterface	Add eth0 [10.132.0.45/23] from ovn-kubernetes
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	Started	Container started
18:33:23	openshift-ingress	multus	maas-default-gateway-openshift-default-55fbd65575-pjmlx	AddedInterface	Add eth0 [10.134.0.57/23] from ovn-kubernetes
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	Created	Container created
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	Created	Container created
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:33:23	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	Started	Container started
18:33:31 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	Unhealthy	Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused
18:33:31 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	Unhealthy	Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused
18:33:31 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	ProbeError	Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused body:
18:33:31 (x8)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	Unhealthy	Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused
18:33:31 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	ProbeError	Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
18:33:31 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	Unhealthy	Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
18:33:31 (x8)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	ProbeError	Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused body:
18:33:31 (x7)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	ProbeError	Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:
18:33:36 (x4)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	Unhealthy	Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:33:37 (x5)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	Unhealthy	Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:33:37 (x5)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	Unhealthy	Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:33:37 (x5)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	Unhealthy	Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:33:37 (x5)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-x9znl	ProbeError	Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:33:38 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-6mddl	ProbeError	Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:33:38 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	ProbeError	Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:33:38 (x6)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-pjmlx	ProbeError	Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:34:17	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	ProbeError	Readiness probe error: Get "http://10.132.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:34:17	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-fnj8g	Unhealthy	Readiness probe failed: Get "http://10.132.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:34:17	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	ProbeError	Readiness probe error: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:34:17	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vlld6	Unhealthy	Readiness probe failed: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:34:21 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	ProbeError	Readiness probe error: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:34:21 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-dfn2b	Unhealthy	Readiness probe failed: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:34:21 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	ProbeError	Readiness probe error: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:34:21 (x2)	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-vdw7q	Unhealthy	Readiness probe failed: Get "http://10.133.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:35:11	openshift-ingress	gateway_labeler_controller	e2e-shared-a-main-84866b	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-main-84866b
18:35:12	openshift-ingress	replicaset-controller	e2e-shared-a-main-84866b-openshift-default-c5ff85555	SuccessfulCreate	Created pod: e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd
18:35:12	openshift-ingress	deployment-controller	e2e-shared-a-main-84866b-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-shared-a-main-84866b-openshift-default-c5ff85555 from 0 to 1
18:35:12	openshift-ingress	multus	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	AddedInterface	Add eth0 [10.134.0.64/23] from ovn-kubernetes
18:35:16	openshift-ingress	replicaset-controller	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767	SuccessfulCreate	Created pod: payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf
18:35:16 (x4)	openshift-ingress	replicaset-controller	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767	FailedCreate	Error creating: pods "payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-main-84866b: serviceaccount "payload-processing-e2e-shared-a-main-84866b" not found
18:35:16	openshift-ingress	replicaset-controller	payload-processing-e2e-shared-a-main-84866b-668c795b9	SuccessfulCreate	Created pod: payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d
18:35:16 (x5)	openshift-ingress	replicaset-controller	payload-processing-e2e-shared-a-main-84866b-668c795b9	FailedCreate	Error creating: pods "payload-processing-e2e-shared-a-main-84866b-668c795b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-main-84866b: serviceaccount "payload-processing-e2e-shared-a-main-84866b" not found
18:35:16	openshift-ingress	deployment-controller	payload-processing-e2e-shared-a-main-84866b	ScalingReplicaSet	Scaled up replica set payload-processing-e2e-shared-a-main-84866b-668c795b9 from 0 to 1
18:35:16	openshift-ingress	deployment-controller	payload-pre-processing-e2e-shared-a-main-84866b	ScalingReplicaSet	Scaled up replica set payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767 from 0 to 1
18:35:17	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf	Pulled	Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
18:35:17	openshift-ingress	kubelet	payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d	Created	Container created
18:35:17	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf	Started	Container started
18:35:17	openshift-ingress	kubelet	payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d	Started	Container started
18:35:17	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf	Created	Container created
18:35:17	openshift-ingress	multus	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf	AddedInterface	Add eth0 [10.134.0.65/23] from ovn-kubernetes
18:35:17	openshift-ingress	kubelet	payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d	Pulled	Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
18:35:17	openshift-ingress	multus	payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d	AddedInterface	Add eth0 [10.134.0.66/23] from ovn-kubernetes
18:35:28	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	ProbeError	Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused body:
18:35:28	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Unhealthy	Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused
18:35:29 (x2)	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Started	Container started
18:35:29 (x2)	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Created	Container created
18:35:29 (x2)	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:35:30	openshift-ingress	replicaset-controller	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57	SuccessfulCreate	Created pod: e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr
18:35:30	openshift-ingress	deployment-controller	e2e-shared-b-main-80c0a4-openshift-default	ScalingReplicaSet	Scaled up replica set e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57 from 0 to 1
18:35:30	openshift-ingress	gateway_labeler_controller	e2e-shared-b-main-80c0a4	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-main-80c0a4
18:35:31	openshift-ingress	multus	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	AddedInterface	Add eth0 [10.133.0.45/23] from ovn-kubernetes
18:35:34 (x5)	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	ProbeError	Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused body:
18:35:34 (x5)	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Unhealthy	Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused
18:35:35	openshift-ingress	deployment-controller	payload-processing-e2e-shared-b-main-80c0a4	ScalingReplicaSet	Scaled up replica set payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9 from 0 to 1
18:35:35	openshift-ingress	deployment-controller	payload-pre-processing-e2e-shared-b-main-80c0a4	ScalingReplicaSet	Scaled up replica set payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b7 from 0 to 1
18:35:35 (x4)	openshift-ingress	replicaset-controller	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b7	FailedCreate	Error creating: pods "payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-main-80c0a4: serviceaccount "payload-processing-e2e-shared-b-main-80c0a4" not found
18:35:35	openshift-ingress	replicaset-controller	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b7	SuccessfulCreate	Created pod: payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq
18:35:35 (x5)	openshift-ingress	replicaset-controller	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9	FailedCreate	Error creating: pods "payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-main-80c0a4: serviceaccount "payload-processing-e2e-shared-b-main-80c0a4" not found
18:35:36	openshift-ingress	replicaset-controller	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9	SuccessfulCreate	Created pod: payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5
18:35:36	openshift-ingress	kubelet	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	Started	Container started
18:35:36	openshift-ingress	multus	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	AddedInterface	Add eth0 [10.134.0.67/23] from ovn-kubernetes
18:35:36	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	Pulled	Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
18:35:36	openshift-ingress	kubelet	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	Created	Container created
18:35:36	openshift-ingress	kubelet	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	Pulled	Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
18:35:36	openshift-ingress	multus	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	AddedInterface	Add eth0 [10.134.0.68/23] from ovn-kubernetes
18:35:36	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	Created	Container created
18:35:36	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	Started	Container started
18:35:45	openshift-ingress	horizontal-pod-autoscaler	e2e-shared-b-main-80c0a4-openshift-default	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:35:45	openshift-ingress	horizontal-pod-autoscaler	e2e-shared-b-main-80c0a4-openshift-default	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:35:49 (x2)	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
18:35:49	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Unhealthy	Readiness probe failed: Get "http://10.133.0.45:15021/healthz/ready": read tcp 10.133.0.2:59636->10.133.0.45:15021: read: connection reset by peer
18:35:49	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	ProbeError	Readiness probe error: Get "http://10.133.0.45:15021/healthz/ready": read tcp 10.133.0.2:59636->10.133.0.45:15021: read: connection reset by peer body:
18:35:49 (x2)	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Started	Container started
18:35:49 (x2)	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Created	Container created
18:35:55 (x5)	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Unhealthy	Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused
18:35:55 (x5)	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	ProbeError	Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:
18:36:12 (x4)	openshift-ingress	horizontal-pod-autoscaler	e2e-shared-a-main-84866b-openshift-default	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:36:12 (x4)	openshift-ingress	horizontal-pod-autoscaler	e2e-shared-a-main-84866b-openshift-default	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
18:36:15 (x2)	openshift-ingress	horizontal-pod-autoscaler	e2e-shared-b-main-80c0a4-openshift-default	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:36:15 (x2)	openshift-ingress	horizontal-pod-autoscaler	e2e-shared-b-main-80c0a4-openshift-default	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
18:36:56	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	ProbeError	Readiness probe error: Get "http://10.134.0.28:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:36:56	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-skbq4	Unhealthy	Readiness probe failed: Get "http://10.134.0.28:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
18:39:09	openshift-ingress	kubelet	maas-default-gateway-openshift-default-55fbd65575-r9hdg	ProbeError	Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
18:40:23	openshift-ingress	kubelet	payload-processing-e2e-shared-a-main-84866b-668c795b9-76v4d	Killing	Stopping container payload-processing
18:40:23	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-a-main-84866b-f7b49c767-2rlwf	Killing	Stopping container payload-pre-processing
18:40:30	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Killing	Stopping container istio-proxy
18:40:35	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	Unhealthy	Readiness probe failed: HTTP probe failed with statuscode: 503
18:40:35	openshift-ingress	kubelet	e2e-shared-a-main-84866b-openshift-default-c5ff85555-jpsdd	ProbeError	Readiness probe error: HTTP probe failed with statuscode: 503 body:
18:40:37	openshift-ingress	kubelet	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	Killing	Stopping container payload-processing
18:40:37	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	Killing	Stopping container payload-pre-processing
18:40:44	openshift-ingress	kubelet	e2e-shared-b-main-80c0a4-openshift-default-578bc6cc57-wzzvr	Killing	Stopping container istio-proxy
18:40:58 (x2)	openshift-ingress	kubelet	payload-processing-e2e-shared-b-main-80c0a4-db94cb7f9-8ztl5	Unhealthy	Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
18:40:58 (x2)	openshift-ingress	kubelet	payload-pre-processing-e2e-shared-b-main-80c0a4-5b5bf974b787bdq	Unhealthy	Readiness probe failed: service unhealthy (responded with "NOT_SERVING")