Events

Time	Namespace	Component	RelatedObject	Reason	Message
16:00:06	openshift-ingress		kube-auth-proxy-68f9447ccb-pjlxq	Scheduled	Successfully assigned openshift-ingress/kube-auth-proxy-68f9447ccb-pjlxq to ip-10-0-131-112.ec2.internal
16:00:14	openshift-ingress		data-science-gateway-data-science-gateway-class-6576859cf4pgwl4	Scheduled	Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4pgwl4 to ip-10-0-131-144.ec2.internal
16:05:12	openshift-ingress		payload-processing-58f86c7798-44sfc	Scheduled	Successfully assigned openshift-ingress/payload-processing-58f86c7798-44sfc to ip-10-0-131-144.ec2.internal
15:54:37	openshift-ingress		router-default-58598f48d7-xkslb	FailedScheduling	0/2 nodes are available: 2 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.
16:00:16	openshift-ingress		istiod-openshift-gateway-94bb8fbfd-mmzxd	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-94bb8fbfd-mmzxd to ip-10-0-131-144.ec2.internal
16:05:12	openshift-ingress		payload-pre-processing-7964457696-jqk7g	Scheduled	Successfully assigned openshift-ingress/payload-pre-processing-7964457696-jqk7g to ip-10-0-131-144.ec2.internal
16:00:06	openshift-ingress		kube-auth-proxy-68f9447ccb-vwnhj	Scheduled	Successfully assigned openshift-ingress/kube-auth-proxy-68f9447ccb-vwnhj to ip-10-0-139-59.ec2.internal
15:55:09	openshift-ingress		router-default-58598f48d7-xkslb	Scheduled	Successfully assigned openshift-ingress/router-default-58598f48d7-xkslb to ip-10-0-139-59.ec2.internal
16:00:08	openshift-ingress		istiod-openshift-gateway-7574459656-d2whb	Scheduled	Successfully assigned openshift-ingress/istiod-openshift-gateway-7574459656-d2whb to ip-10-0-131-112.ec2.internal
16:02:59	openshift-ingress		maas-default-gateway-openshift-default-56654dcf7-flxnb	Scheduled	Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-56654dcf7-flxnb to ip-10-0-131-144.ec2.internal
15:48:26	openshift-ingress	deployment-controller	router-default	ScalingReplicaSet	Scaled up replica set router-default-58598f48d7 from 0 to 1
15:48:27	openshift-ingress	service-controller	router-default	UnAvailableLoadBalancer	There are no available nodes for LoadBalancer
15:48:27	openshift-ingress	service-controller	router-default	EnsuringLoadBalancer	Ensuring load balancer
15:48:32	openshift-ingress	service-controller	router-default	EnsuredLoadBalancer	Ensured load balancer
15:48:37 (x12)	openshift-ingress	replicaset-controller	router-default-58598f48d7	FailedCreate	Error creating: pods "router-default-58598f48d7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
15:51:53 (x16)	openshift-ingress	replicaset-controller	router-default-58598f48d7	FailedCreate	Error creating: pods "router-default-58598f48d7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
15:54:37	openshift-ingress	replicaset-controller	router-default-58598f48d7	SuccessfulCreate	Created pod: router-default-58598f48d7-xkslb
15:54:47 (x3)	openshift-ingress	service-controller	router-default	UpdatedLoadBalancer	Updated load balancer with new hosts
15:55:25 (x6)	openshift-ingress	kubelet	router-default-58598f48d7-xkslb	FailedMount	MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
15:55:25 (x6)	openshift-ingress	kubelet	router-default-58598f48d7-xkslb	FailedMount	MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
15:55:41	openshift-ingress	multus	router-default-58598f48d7-xkslb	AddedInterface	Add eth0 [10.132.0.11/23] from ovn-kubernetes
15:55:41	openshift-ingress	kubelet	router-default-58598f48d7-xkslb	Pulled	Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:d0a4ad778508b54686000a2e9088f085a174aa26c504abe9f8ffe8cab7fdd077" already present on machine
15:55:42	openshift-ingress	kubelet	router-default-58598f48d7-xkslb	Started	Started container router
15:55:42	openshift-ingress	kubelet	router-default-58598f48d7-xkslb	Created	Created container: router
16:00:06 (x5)	openshift-ingress	replicaset-controller	kube-auth-proxy-68f9447ccb	FailedCreate	Error creating: pods "kube-auth-proxy-68f9447ccb-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found
16:00:06	openshift-ingress	replicaset-controller	kube-auth-proxy-68f9447ccb	SuccessfulCreate	Created pod: kube-auth-proxy-68f9447ccb-pjlxq
16:00:06	openshift-ingress	deployment-controller	kube-auth-proxy	ScalingReplicaSet	Scaled up replica set kube-auth-proxy-68f9447ccb from 0 to 2
16:00:06	openshift-ingress	replicaset-controller	kube-auth-proxy-68f9447ccb	SuccessfulCreate	Created pod: kube-auth-proxy-68f9447ccb-vwnhj
16:00:07	openshift-ingress	multus	kube-auth-proxy-68f9447ccb-vwnhj	AddedInterface	Add eth0 [10.132.0.34/23] from ovn-kubernetes
16:00:07	openshift-ingress	multus	kube-auth-proxy-68f9447ccb-pjlxq	AddedInterface	Add eth0 [10.134.0.18/23] from ovn-kubernetes
16:00:07	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-pjlxq	Pulling	Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"
16:00:07	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-vwnhj	Pulling	Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"
16:00:08	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-7574459656 from 0 to 1
16:00:08	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7574459656	SuccessfulCreate	Created pod: istiod-openshift-gateway-7574459656-d2whb
16:00:08	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-d2whb	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"
16:00:08	openshift-ingress	multus	istiod-openshift-gateway-7574459656-d2whb	AddedInterface	Add eth0 [10.134.0.21/23] from ovn-kubernetes
16:00:10	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-vwnhj	Created	Created container: kube-auth-proxy
16:00:10	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-vwnhj	Pulled	Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 3.497s (3.497s including waiting). Image size: 265120132 bytes.
16:00:10	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-vwnhj	Started	Started container kube-auth-proxy
16:00:12	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-pjlxq	Started	Started container kube-auth-proxy
16:00:12	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-pjlxq	Created	Created container: kube-auth-proxy
16:00:12	openshift-ingress	kubelet	kube-auth-proxy-68f9447ccb-pjlxq	Pulled	Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 4.717s (4.717s including waiting). Image size: 265120132 bytes.
16:00:13	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-d2whb	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 4.949s (4.949s including waiting). Image size: 181034515 bytes.
16:00:13	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-d2whb	Started	Started container discovery
16:00:13	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-d2whb	Created	Created container: discovery
16:00:14	openshift-ingress	replicaset-controller	data-science-gateway-data-science-gateway-class-6576859cf4	SuccessfulCreate	Created pod: data-science-gateway-data-science-gateway-class-6576859cf4pgwl4
16:00:14	openshift-ingress	deployment-controller	data-science-gateway-data-science-gateway-class	ScalingReplicaSet	Scaled up replica set data-science-gateway-data-science-gateway-class-6576859cf4 from 0 to 1
16:00:15	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4pgwl4	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8"
16:00:15	openshift-ingress	multus	data-science-gateway-data-science-gateway-class-6576859cf4pgwl4	AddedInterface	Add eth0 [10.133.0.22/23] from ovn-kubernetes
16:00:16	openshift-ingress	replicaset-controller	istiod-openshift-gateway-94bb8fbfd	SuccessfulCreate	Created pod: istiod-openshift-gateway-94bb8fbfd-mmzxd
16:00:16	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled up replica set istiod-openshift-gateway-94bb8fbfd from 0 to 1
16:00:17	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-mmzxd	Pulling	Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9"
16:00:17	openshift-ingress	multus	istiod-openshift-gateway-94bb8fbfd-mmzxd	AddedInterface	Add eth0 [10.133.0.23/23] from ovn-kubernetes
16:00:20	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-mmzxd	Created	Created container: discovery
16:00:20	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-mmzxd	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:0850242436e88f7d82f0f2126de064c7e0f09844f31d8ff0f53dc8d3908075d9" in 3.298s (3.298s including waiting). Image size: 181034515 bytes.
16:00:20	openshift-ingress	kubelet	istiod-openshift-gateway-94bb8fbfd-mmzxd	Started	Started container discovery
16:00:20	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4pgwl4	Started	Started container istio-proxy
16:00:20	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4pgwl4	Created	Created container: istio-proxy
16:00:20	openshift-ingress	kubelet	data-science-gateway-data-science-gateway-class-6576859cf4pgwl4	Pulled	Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" in 5.181s (5.181s including waiting). Image size: 181251556 bytes.
16:00:21	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
16:00:21	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
16:00:22	openshift-ingress	deployment-controller	istiod-openshift-gateway	ScalingReplicaSet	Scaled down replica set istiod-openshift-gateway-7574459656 from 1 to 0
16:00:22	openshift-ingress	replicaset-controller	istiod-openshift-gateway-7574459656	SuccessfulDelete	Deleted pod: istiod-openshift-gateway-7574459656-d2whb
16:00:22	openshift-ingress	kubelet	istiod-openshift-gateway-7574459656-d2whb	Killing	Stopping container discovery
16:00:36	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
16:00:36	openshift-ingress	horizontal-pod-autoscaler	kube-auth-proxy	FailedGetResourceMetric	failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
16:01:08 (x4)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedGetResourceMetric	failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
16:01:08 (x4)	openshift-ingress	horizontal-pod-autoscaler	istiod-openshift-gateway	FailedComputeMetricsReplicas	invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
16:02:58	openshift-ingress	gateway_labeler_controller	maas-default-gateway	AddedLabel	Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway
16:02:59	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-flxnb	Started	Started container istio-proxy
16:02:59	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-flxnb	Created	Created container: istio-proxy
16:02:59	openshift-ingress	deployment-controller	maas-default-gateway-openshift-default	ScalingReplicaSet	Scaled up replica set maas-default-gateway-openshift-default-56654dcf7 from 0 to 1
16:02:59	openshift-ingress	kubelet	maas-default-gateway-openshift-default-56654dcf7-flxnb	Pulled	Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:7d15cebf9b62f3f235c0eab5158ac8ff2fda86a1d193490dc94c301402c99da8" already present on machine
16:02:59	openshift-ingress	replicaset-controller	maas-default-gateway-openshift-default-56654dcf7	SuccessfulCreate	Created pod: maas-default-gateway-openshift-default-56654dcf7-flxnb
16:02:59	openshift-ingress	multus	maas-default-gateway-openshift-default-56654dcf7-flxnb	AddedInterface	Add eth0 [10.133.0.25/23] from ovn-kubernetes
16:05:11 (x4)	openshift-ingress	replicaset-controller	payload-pre-processing-7964457696	FailedCreate	Error creating: pods "payload-pre-processing-7964457696-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
16:05:11	openshift-ingress	deployment-controller	payload-pre-processing	ScalingReplicaSet	Scaled up replica set payload-pre-processing-7964457696 from 0 to 1
16:05:11	openshift-ingress	deployment-controller	payload-processing	ScalingReplicaSet	Scaled up replica set payload-processing-58f86c7798 from 0 to 1
16:05:11 (x5)	openshift-ingress	replicaset-controller	payload-processing-58f86c7798	FailedCreate	Error creating: pods "payload-processing-58f86c7798-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
16:05:12	openshift-ingress	replicaset-controller	payload-pre-processing-7964457696	SuccessfulCreate	Created pod: payload-pre-processing-7964457696-jqk7g
16:05:12	openshift-ingress	kubelet	payload-processing-58f86c7798-44sfc	Pulling	Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0"
16:05:12	openshift-ingress	multus	payload-processing-58f86c7798-44sfc	AddedInterface	Add eth0 [10.133.0.30/23] from ovn-kubernetes
16:05:12	openshift-ingress	replicaset-controller	payload-processing-58f86c7798	SuccessfulCreate	Created pod: payload-processing-58f86c7798-44sfc
16:05:12	openshift-ingress	kubelet	payload-pre-processing-7964457696-jqk7g	Pulling	Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0"
16:05:12	openshift-ingress	multus	payload-pre-processing-7964457696-jqk7g	AddedInterface	Add eth0 [10.133.0.29/23] from ovn-kubernetes
16:05:14	openshift-ingress	kubelet	payload-processing-58f86c7798-44sfc	Created	Created container: payload-processing
16:05:14	openshift-ingress	kubelet	payload-processing-58f86c7798-44sfc	Pulled	Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0" in 1.808s (1.808s including waiting). Image size: 165432098 bytes.
16:05:14	openshift-ingress	kubelet	payload-processing-58f86c7798-44sfc	Started	Started container payload-processing
16:05:14	openshift-ingress	kubelet	payload-pre-processing-7964457696-jqk7g	Started	Started container payload-pre-processing
16:05:14	openshift-ingress	kubelet	payload-pre-processing-7964457696-jqk7g	Created	Created container: payload-pre-processing
16:05:14	openshift-ingress	kubelet	payload-pre-processing-7964457696-jqk7g	Pulled	Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:dc31b949d8c5ea8610b5c10ef53e2a199e00f0e0" in 1.848s (1.848s including waiting). Image size: 165432098 bytes.