Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-5cc6cd444b-9z7ng

Scheduled

Successfully assigned openshift-ingress/payload-processing-5cc6cd444b-9z7ng to ip-10-0-140-149.ec2.internal

openshift-ingress

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Scheduled

Successfully assigned openshift-ingress/e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz to ip-10-0-141-133.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-75lrr

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-75lrr to ip-10-0-131-12.ec2.internal

openshift-ingress

kube-auth-proxy-66d67d68d4-d8wjk

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-66d67d68d4-d8wjk to ip-10-0-140-149.ec2.internal

openshift-ingress

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Scheduled

Successfully assigned openshift-ingress/e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m to ip-10-0-131-12.ec2.internal

openshift-ingress

kube-auth-proxy-66d67d68d4-r6ngq

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-66d67d68d4-r6ngq to ip-10-0-141-133.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv to ip-10-0-131-12.ec2.internal

openshift-ingress

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x to ip-10-0-141-133.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-v8jkd to ip-10-0-141-133.ec2.internal

openshift-ingress

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6 to ip-10-0-131-12.ec2.internal

openshift-ingress

payload-pre-processing-687dbb6cf6-p6w7n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-687dbb6cf6-p6w7n to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4 to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx to ip-10-0-141-133.ec2.internal

openshift-ingress

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8 to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7 to ip-10-0-141-133.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class

ClusterIPNotAllocated

Cluster IP [IPv4]: 172.31.1.194 is not allocated; repairing

openshift-ingress

data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799c9pz22 to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r to ip-10-0-141-133.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-tbr6c

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-tbr6c to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb to ip-10-0-141-133.ec2.internal

openshift-ingress

router-default-78dd4996b7-bzkqg

Scheduled

Successfully assigned openshift-ingress/router-default-78dd4996b7-bzkqg to ip-10-0-131-12.ec2.internal

openshift-ingress

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Scheduled

Successfully assigned openshift-ingress/e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb to ip-10-0-141-133.ec2.internal

openshift-ingress

payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x to ip-10-0-141-133.ec2.internal

openshift-ingress

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Scheduled

Successfully assigned openshift-ingress/e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7 to ip-10-0-131-12.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-78dd4996b7 from 0 to 1
(x11)

openshift-ingress

replicaset-controller

router-default-78dd4996b7

FailedCreate

Error creating: pods "router-default-78dd4996b7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-78dd4996b7

FailedCreate

Error creating: pods "router-default-78dd4996b7-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-78dd4996b7

SuccessfulCreate

Created pod: router-default-78dd4996b7-bzkqg
(x7)

openshift-ingress

kubelet

router-default-78dd4996b7-bzkqg

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x7)

openshift-ingress

kubelet

router-default-78dd4996b7-bzkqg

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-78dd4996b7-bzkqg

Started

Started container router

openshift-ingress

kubelet

router-default-78dd4996b7-bzkqg

Created

Created container: router

openshift-ingress

multus

router-default-78dd4996b7-bzkqg

AddedInterface

Add eth0 [10.133.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-78dd4996b7-bzkqg

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559" already present on machine

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-66d67d68d4 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-66d67d68d4

SuccessfulCreate

Created pod: kube-auth-proxy-66d67d68d4-r6ngq

openshift-ingress

replicaset-controller

kube-auth-proxy-66d67d68d4

SuccessfulCreate

Created pod: kube-auth-proxy-66d67d68d4-d8wjk

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-r6ngq

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-66d67d68d4-r6ngq

AddedInterface

Add eth0 [10.132.0.15/23] from ovn-kubernetes
(x3)

openshift-ingress

replicaset-controller

kube-auth-proxy-66d67d68d4

FailedCreate

Error creating: pods "kube-auth-proxy-66d67d68d4-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-d8wjk

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-66d67d68d4-d8wjk

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-tbr6c

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-d8wjk

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-d8wjk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.184s (2.184s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-d8wjk

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-r6ngq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 4.292s (4.292s including waiting). Image size: 265120132 bytes.

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-tbr6c

AddedInterface

Add eth0 [10.132.0.18/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-r6ngq

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-66d67d68d4-r6ngq

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.592s (2.592s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-fddd6c88c from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-75lrr

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-fddd6c88c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-68cf4d799c from 0 to 1

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

AddedInterface

Add eth0 [10.132.0.19/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-68cf4d799c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-75lrr

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-75lrr

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.453s (2.453s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.529s (2.529s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799c9pz22

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-fddd6c88c

SuccessfulDelete

Deleted pod: data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled down replica set data-science-gateway-data-science-gateway-class-fddd6c88c from 1 to 0
(x3)

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

ProbeError

Startup probe error: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Unhealthy

Startup probe failed: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-fddd6c88c-rtlhv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-75lrr

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 7.669s (7.669s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-75lrr

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-75lrr

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-tbr6c

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-tbr6c

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5ccb968c66

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5ccb968c66-v8jkd

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5ccb968c66 from 0 to 1

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

multus

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-5cc6cd444b-9z7ng

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-687dbb6cf6-p6w7n

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-5cc6cd444b

SuccessfulCreate

Created pod: payload-processing-5cc6cd444b-9z7ng

openshift-ingress

replicaset-controller

payload-pre-processing-687dbb6cf6

SuccessfulCreate

Created pod: payload-pre-processing-687dbb6cf6-p6w7n
(x5)

openshift-ingress

replicaset-controller

payload-processing-5cc6cd444b

FailedCreate

Error creating: pods "payload-processing-5cc6cd444b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5cc6cd444b from 0 to 1

openshift-ingress

kubelet

payload-processing-5cc6cd444b-9z7ng

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-p6w7n

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-687dbb6cf6 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-687dbb6cf6

FailedCreate

Error creating: pods "payload-pre-processing-687dbb6cf6-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-processing-5cc6cd444b-9z7ng

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.037s (2.037s including waiting). Image size: 165776054 bytes.

openshift-ingress

kubelet

payload-processing-5cc6cd444b-9z7ng

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5cc6cd444b-9z7ng

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-p6w7n

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.195s (3.195s including waiting). Image size: 165776054 bytes.

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-p6w7n

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-p6w7n

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

ProbeError

Startup probe error: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-v8jkd

Unhealthy

Startup probe failed: Get "http://10.132.0.32:15021/healthz/ready": dial tcp 10.132.0.32:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-ait-ee2b828d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ee2b828d

openshift-ingress

deployment-controller

e2e-ait-ee2b828d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-ee2b828d-openshift-default-5bc675897b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-ee2b828d-openshift-default-5bc675897b

SuccessfulCreate

Created pod: e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

openshift-ingress

multus

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-ee2b828d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ee2b828d: serviceaccount "payload-processing-e2e-ait-ee2b828d" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-ee2b828d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-ee2b828d-7f8f447c88 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ee2b828d-7f8f447c88

FailedCreate

Error creating: pods "payload-processing-e2e-ait-ee2b828d-7f8f447c88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ee2b828d: serviceaccount "payload-processing-e2e-ait-ee2b828d" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ee2b828d-7f8f447c88

SuccessfulCreate

Created pod: payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

openshift-ingress

kubelet

payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Unhealthy

Readiness probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

ProbeError

Readiness probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ait-ee2b828d-7f8f447c88-9phmb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee2b828d-7f847c6dbd-rgvqb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-ee2b828d-openshift-default-5bc675897b-ngsvz

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd

SuccessfulCreate

Created pod: e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

openshift-ingress

deployment-controller

e2e-migrate-47d0d404-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-47d0d404-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-47d0d404-gw

openshift-ingress

multus

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-47d0d404

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-47d0d404-68f98569cc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-47d0d404

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-47d0d404-c489cc848 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-47d0d404-68f98569cc

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-47d0d404-68f98569cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-47d0d404: serviceaccount "payload-processing-e2e-migrate-47d0d404" not found

openshift-ingress

kubelet

payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-47d0d404-68f98569cc

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

openshift-ingress

kubelet

payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-47d0d404-c489cc848

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-47d0d404-c489cc848

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-47d0d404-c489cc848-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-47d0d404: serviceaccount "payload-processing-e2e-migrate-47d0d404" not found

openshift-ingress

multus

payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Unhealthy

Readiness probe failed: Get "http://10.133.0.41:15021/healthz/ready": read tcp 10.133.0.2:54310->10.133.0.41:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

ProbeError

Readiness probe error: Get "http://10.133.0.41:15021/healthz/ready": read tcp 10.133.0.2:54310->10.133.0.41:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-migrate-47d0d404-68f98569cc-x956r

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-47d0d404-c489cc848-tklj8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-47d0d404-gw-openshift-default-84f5f89bdd-l5mz6

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-a1550b4d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a1550b4d

openshift-ingress

replicaset-controller

e2e-ait-a1550b4d-openshift-default-7c995b4956

SuccessfulCreate

Created pod: e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

openshift-ingress

multus

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ait-a1550b4d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-a1550b4d-openshift-default-7c995b4956 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-a1550b4d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-a1550b4d-58fb4b6d68 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-a1550b4d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-a1550b4d-58fb4b6d68

SuccessfulCreate

Created pod: payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-a1550b4d-58fb4b6d68

FailedCreate

Error creating: pods "payload-processing-e2e-ait-a1550b4d-58fb4b6d68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a1550b4d: serviceaccount "payload-processing-e2e-ait-a1550b4d" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a1550b4d: serviceaccount "payload-processing-e2e-ait-a1550b4d" not found

openshift-ingress

kubelet

payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Unhealthy

Readiness probe failed: Get "http://10.132.0.58:15021/healthz/ready": read tcp 10.132.0.2:34390->10.132.0.58:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

ProbeError

Readiness probe error: Get "http://10.132.0.58:15021/healthz/ready": read tcp 10.132.0.2:34390->10.132.0.58:15021: read: connection reset by peer body:
(x4)

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ait-a1550b4d-58fb4b6d68-dfg7x

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a1550b4d-84d8c794dd-gqvd4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-a1550b4d-openshift-default-7c995b4956-h4fkz

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-84b622a4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-84b622a4-openshift-default-7bd54bcdfb from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-84b622a4-openshift-default-7bd54bcdfb

SuccessfulCreate

Created pod: e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

openshift-ingress

multus

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-derive-84b622a4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-84b622a4

openshift-ingress

deployment-controller

payload-processing-e2e-derive-84b622a4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-84b622a4-b486975b6 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-84b622a4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-84b622a4-b486975b6

SuccessfulCreate

Created pod: payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-84b622a4-b486975b6

FailedCreate

Error creating: pods "payload-processing-e2e-derive-84b622a4-b486975b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-84b622a4: serviceaccount "payload-processing-e2e-derive-84b622a4" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-84b622a4: serviceaccount "payload-processing-e2e-derive-84b622a4" not found

openshift-ingress

multus

payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Unhealthy

Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": read tcp 10.133.0.2:43174->10.133.0.44:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

ProbeError

Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": read tcp 10.133.0.2:43174->10.133.0.44:15021: read: connection reset by peer body:
(x3)

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-84b622a4-6d5b7d4576-z98cx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-84b622a4-b486975b6-g8nvc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-84b622a4-openshift-default-7bd54bcdfb-rk5f7

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-mt-c811f4d5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-c811f4d5-openshift-default-67bb668d6d from 0 to 1

openshift-ingress

multus

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-c811f4d5-openshift-default-67bb668d6d

SuccessfulCreate

Created pod: e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

openshift-ingress

gateway_labeler_controller

e2e-mt-c811f4d5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-c811f4d5
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c811f4d5: serviceaccount "payload-processing-e2e-mt-c811f4d5" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-c811f4d5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c811f4d5-64b89dcbbb

FailedCreate

Error creating: pods "payload-processing-e2e-mt-c811f4d5-64b89dcbbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c811f4d5: serviceaccount "payload-processing-e2e-mt-c811f4d5" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-c811f4d5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-c811f4d5-64b89dcbbb from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c811f4d5-64b89dcbbb

SuccessfulCreate

Created pod: payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

openshift-ingress

multus

payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Unhealthy

Readiness probe failed: dial tcp 10.132.0.67:9004: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Unhealthy

Liveness probe failed: dial tcp 10.132.0.67:9004: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-mt-c811f4d5-64b89dcbbb-2s5bg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c811f4d5-5fc64798bd-swnw7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-c811f4d5-openshift-default-67bb668d6d-l796x

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

gateway_labeler_controller

e2e-mt-e61f3ca0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-e61f3ca0

openshift-ingress

deployment-controller

e2e-mt-e61f3ca0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-e61f3ca0-openshift-default-6f5748cb68 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-e61f3ca0-openshift-default-6f5748cb68

SuccessfulCreate

Created pod: e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

openshift-ingress

multus

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-e61f3ca0-759bbc8954

SuccessfulCreate

Created pod: payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-e61f3ca0-759bbc8954

FailedCreate

Error creating: pods "payload-processing-e2e-mt-e61f3ca0-759bbc8954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-e61f3ca0: serviceaccount "payload-processing-e2e-mt-e61f3ca0" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-e61f3ca0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-e61f3ca0-747fd69496 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-e61f3ca0: serviceaccount "payload-processing-e2e-mt-e61f3ca0" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-e61f3ca0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-e61f3ca0-759bbc8954 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l

Started

Started container payload-processing

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Unhealthy

Readiness probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

ProbeError

Readiness probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Created

Created container: istio-proxy
(x3)

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Unhealthy

Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused
(x3)

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

ProbeError

Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-mt-e61f3ca0-759bbc8954-gd92l

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-e61f3ca0

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-e61f3ca0": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-e61f3ca0, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f938b1df-bb78-4240-8c56-4be89e7c0b26, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-e61f3ca0-747fd69496-kbz9r

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-e61f3ca0-openshift-default-6f5748cb68-btt2m

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-inf-a-6c969d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-6c969d-openshift-default-5dbc846d4d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-a-6c969d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-6c969d

openshift-ingress

replicaset-controller

e2e-inf-a-6c969d-openshift-default-5dbc846d4d

SuccessfulCreate

Created pod: e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

openshift-ingress

multus

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n

Created

Created container: payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-6c969d-66d8b8656

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-6c969d-66d8b8656-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6c969d: serviceaccount "payload-processing-e2e-inf-a-6c969d" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-6c969d-66d8b8656

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-6c969d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-6c969d-66d8b8656 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6c969d: serviceaccount "payload-processing-e2e-inf-a-6c969d" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-6c969d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x3)

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x3)

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-6c969d-c44fcf78d-k2f5n

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-6c969d-66d8b8656-5l4rb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-a-6c969d-openshift-default-5dbc846d4d-5f8cz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: