Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-noten-836d37-openshift-default-58bc769474-4gtk6

Scheduled

Successfully assigned openshift-ingress/e2e-noten-836d37-openshift-default-58bc769474-4gtk6 to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7 to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

Scheduled

Successfully assigned openshift-ingress/e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7 to ip-10-0-129-222.ec2.internal

openshift-ingress

kube-auth-proxy-5cc5ff74fb-xl5vx

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5cc5ff74fb-xl5vx to ip-10-0-135-107.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

Scheduled

Successfully assigned openshift-ingress/e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6 to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Scheduled

Successfully assigned openshift-ingress/e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk to ip-10-0-129-222.ec2.internal

openshift-ingress

kube-auth-proxy-5cc5ff74fb-8j6n9

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5cc5ff74fb-8j6n9 to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt to ip-10-0-129-222.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44 to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9 to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

Scheduled

Successfully assigned openshift-ingress/e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-816605-openshift-default-76dd9c445d-259dd to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf to ip-10-0-133-50.ec2.internal

openshift-ingress

router-default-5946c9f774-tk9pg

Scheduled

Successfully assigned openshift-ingress/router-default-5946c9f774-tk9pg to ip-10-0-135-107.ec2.internal

openshift-ingress

payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5 to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2 to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

Scheduled

Successfully assigned openshift-ingress/e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-jmnn9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-jmnn9 to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-aba078c2-84c9648747-khpz7 to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2 to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7 to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc to ip-10-0-129-222.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-2t8jt to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Scheduled

Successfully assigned openshift-ingress/e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j to ip-10-0-129-222.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz to ip-10-0-129-222.ec2.internal

openshift-ingress

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm to ip-10-0-133-50.ec2.internal

openshift-ingress

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf to ip-10-0-133-50.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-wcfks

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-wcfks to ip-10-0-133-50.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-8dn7w

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-8dn7w to ip-10-0-133-50.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-l57cr

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-l57cr to ip-10-0-135-107.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-5946c9f774 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x14)

openshift-ingress

replicaset-controller

router-default-5946c9f774

FailedCreate

Error creating: pods "router-default-5946c9f774-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-5946c9f774

FailedCreate

Error creating: pods "router-default-5946c9f774-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-5946c9f774

SuccessfulCreate

Created pod: router-default-5946c9f774-tk9pg
(x5)

openshift-ingress

kubelet

router-default-5946c9f774-tk9pg

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-5946c9f774-tk9pg

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-5946c9f774-tk9pg

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

multus

router-default-5946c9f774-tk9pg

AddedInterface

Add eth0 [10.133.0.11/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-5946c9f774-tk9pg

Created

Created container: router

openshift-ingress

kubelet

router-default-5946c9f774-tk9pg

Started

Started container router

openshift-ingress

replicaset-controller

kube-auth-proxy-5cc5ff74fb

SuccessfulCreate

Created pod: kube-auth-proxy-5cc5ff74fb-8j6n9

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-5cc5ff74fb from 0 to 2

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-xl5vx

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

kube-auth-proxy-5cc5ff74fb

FailedCreate

Error creating: pods "kube-auth-proxy-5cc5ff74fb-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

multus

kube-auth-proxy-5cc5ff74fb-8j6n9

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

kube-auth-proxy-5cc5ff74fb

SuccessfulCreate

Created pod: kube-auth-proxy-5cc5ff74fb-xl5vx

openshift-ingress

multus

kube-auth-proxy-5cc5ff74fb-xl5vx

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-8j6n9

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-l57cr

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-xl5vx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.794s (2.794s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-l57cr

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-xl5vx

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-xl5vx

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-8j6n9

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-8j6n9

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5cc5ff74fb-8j6n9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 3.999s (3.999s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.537s (2.537s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

Created

Created container: discovery

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-4fx44

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.909s (2.909s including waiting). Image size: 181318953 bytes.

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-8dn7w

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-8dn7w

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-8dn7w

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-8dn7w

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-8dn7w

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.233s (2.233s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-8dn7w

Started

Started container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-l57cr

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-l57cr

Killing

Stopping container discovery
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-2t8jt

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.848s (2.848s including waiting). Image size: 181318953 bytes.
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-jmnn9

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-wcfks

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-jmnn9

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-jmnn9

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wcfks

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-5b54b7fbb7-wcfks

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-jmnn9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.829s (1.829s including waiting). Image size: 166449337 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wcfks

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wcfks

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wcfks

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.877s (1.877s including waiting). Image size: 166449337 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-jmnn9

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-jmnn9

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

Unhealthy

Startup probe failed: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-2t8jt

ProbeError

Startup probe error: Get "http://10.132.0.31:15021/healthz/ready": dial tcp 10.132.0.31:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-inf-a-cef92e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-cef92e-openshift-default-685dfdf9fd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-6ba087

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-6ba087

openshift-ingress

replicaset-controller

e2e-inf-a-cef92e-openshift-default-685dfdf9fd

SuccessfulCreate

Created pod: e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-651569

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-651569

openshift-ingress

gateway_labeler_controller

e2e-inf-a-cef92e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-cef92e

openshift-ingress

replicaset-controller

e2e-worker-w4-651569-openshift-default-cdcc9bdb4

SuccessfulCreate

Created pod: e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

openshift-ingress

multus

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-651569-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-651569-openshift-default-cdcc9bdb4 from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w0-6ba087-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-6ba087-openshift-default-85464b8f68 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w0-6ba087-openshift-default-85464b8f68

SuccessfulCreate

Created pod: e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-cef92e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-cef92e-65546b8d85 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-651569

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-651569-5b8f855699 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-651569-5b8f855699

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-651569-5b8f855699

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-651569-5b8f855699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-651569: serviceaccount "payload-processing-e2e-worker-w4-651569" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-cef92e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-cef92e-5678c5544b from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-651569

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb from 0 to 1

openshift-ingress

multus

payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-651569: serviceaccount "payload-processing-e2e-worker-w4-651569" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-cef92e-65546b8d85

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-cef92e-65546b8d85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-cef92e: serviceaccount "payload-processing-e2e-inf-a-cef92e" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-cef92e: serviceaccount "payload-processing-e2e-inf-a-cef92e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

openshift-ingress

multus

payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-cef92e-65546b8d85

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

openshift-ingress

deployment-controller

e2e-autores-dc7dcd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-dc7dcd-openshift-default-5958f7974d from 0 to 1

openshift-ingress

multus

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-6ba087

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-6ba087: serviceaccount "payload-processing-e2e-worker-w0-6ba087" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

gateway_labeler_controller

e2e-autores-dc7dcd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-dc7dcd

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-6ba087

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-6ba087: serviceaccount "payload-processing-e2e-worker-w0-6ba087" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

e2e-autores-dc7dcd-openshift-default-5958f7974d

SuccessfulCreate

Created pod: e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

Started

Started container payload-processing

openshift-ingress

multus

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ait-0bc9acd8-openshift-default-744d69d584

SuccessfulCreate

Created pod: e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

openshift-ingress

deployment-controller

e2e-ait-0bc9acd8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-0bc9acd8-openshift-default-744d69d584 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-0bc9acd8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-0bc9acd8

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.631s (2.631s including waiting). Image size: 166449337 bytes.

openshift-ingress

multus

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-dc7dcd-57499fcd99

FailedCreate

Error creating: pods "payload-processing-e2e-autores-dc7dcd-57499fcd99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-dc7dcd: serviceaccount "payload-processing-e2e-autores-dc7dcd" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-dc7dcd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-dc7dcd-846dffff5d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-autores-dc7dcd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-dc7dcd-57499fcd99 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-dc7dcd: serviceaccount "payload-processing-e2e-autores-dc7dcd" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-dc7dcd-57499fcd99

SuccessfulCreate

Created pod: payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-0bc9acd8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-0bc9acd8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-0bc9acd8-7777f5d8d7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7

FailedCreate

Error creating: pods "payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0bc9acd8: serviceaccount "payload-processing-e2e-ait-0bc9acd8" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

openshift-ingress

multus

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7

SuccessfulCreate

Created pod: payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0bc9acd8: serviceaccount "payload-processing-e2e-ait-0bc9acd8" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

Started

Started container istio-proxy

openshift-ingress

deployment-controller

e2e-inf-b-891e67-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-891e67-openshift-default-564fcdf9fb from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

Unhealthy

Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

ProbeError

Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-inf-b-891e67

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-891e67
(x7)

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-inf-b-891e67-openshift-default-564fcdf9fb

SuccessfulCreate

Created pod: e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

openshift-ingress

multus

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

ProbeError

Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-891e67

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-891e67: serviceaccount "payload-processing-e2e-inf-b-891e67" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-891e67-55d8c8cf55

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-891e67

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-891e67-55d8c8cf55 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Unhealthy

Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-891e67-55d8c8cf55

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-891e67-55d8c8cf55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-891e67: serviceaccount "payload-processing-e2e-inf-b-891e67" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

ProbeError

Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

Unhealthy

Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

ProbeError

Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

Unhealthy

Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ait-0bc9acd8-7777f5d8d7-4mzc2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0bc9acd8-56d475fd66-6gsbn

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

ProbeError

Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ait-0bc9acd8-openshift-default-744d69d584-x6fhm

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-migrate-1d3c7cc9-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548 from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

Unhealthy

Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-migrate-1d3c7cc9-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-1d3c7cc9-gw

openshift-ingress

multus

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548

SuccessfulCreate

Created pod: e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-1d3c7cc9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-1d3c7cc9-fd48988c8 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-1d3c7cc9: serviceaccount "payload-processing-e2e-migrate-1d3c7cc9" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-1d3c7cc9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-1d3c7cc9: serviceaccount "payload-processing-e2e-migrate-1d3c7cc9" not found

openshift-ingress

kubelet

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-migrate-1d3c7cc9-fd48988c8-65qvp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-1d3c7cc9-669cdd9b6b-mnxqr

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-ait-aba078c2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-aba078c2-openshift-default-6c4db56c6d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-aba078c2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-aba078c2

openshift-ingress

kubelet

e2e-migrate-1d3c7cc9-gw-openshift-default-79869c5548-mqlm9

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-aba078c2-openshift-default-6c4db56c6d

SuccessfulCreate

Created pod: e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

openshift-ingress

multus

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-aba078c2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-aba078c2-5f89d47446 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-aba078c2-5f89d47446

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-aba078c2-5f89d47446-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-aba078c2: serviceaccount "payload-processing-e2e-ait-aba078c2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-aba078c2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-aba078c2-84c9648747 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-aba078c2-84c9648747

SuccessfulCreate

Created pod: payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-aba078c2-5f89d47446

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-aba078c2-84c9648747

FailedCreate

Error creating: pods "payload-processing-e2e-ait-aba078c2-84c9648747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-aba078c2: serviceaccount "payload-processing-e2e-ait-aba078c2" not found

openshift-ingress

multus

payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-aba078c2-5f89d47446-dv6fv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-aba078c2-84c9648747-khpz7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-aba078c2-openshift-default-6c4db56c6d-xr2wt

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-39ea7c6f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-39ea7c6f-openshift-default-566bd9c7d4 from 0 to 1

openshift-ingress

multus

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-derive-39ea7c6f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-39ea7c6f

openshift-ingress

replicaset-controller

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4

SuccessfulCreate

Created pod: e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-dc7dcd-846dffff5d-zffpq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-dc7dcd-57499fcd99-bhxnd

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-derive-39ea7c6f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-39ea7c6f-fd968b857 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-39ea7c6f-fd968b857

SuccessfulCreate

Created pod: payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-39ea7c6f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-39ea7c6f-69f5574754 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-39ea7c6f-fd968b857

FailedCreate

Error creating: pods "payload-processing-e2e-derive-39ea7c6f-fd968b857-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-39ea7c6f: serviceaccount "payload-processing-e2e-derive-39ea7c6f" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-39ea7c6f: serviceaccount "payload-processing-e2e-derive-39ea7c6f" not found

openshift-ingress

kubelet

payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-autores-dc7dcd-openshift-default-5958f7974d-457sp

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-explicit-8d451c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-8d451c-openshift-default-7b5868fcdd from 0 to 1

openshift-ingress

multus

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-explicit-8d451c-openshift-default-7b5868fcdd

SuccessfulCreate

Created pod: e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

openshift-ingress

gateway_labeler_controller

e2e-explicit-8d451c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-8d451c

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

ProbeError

Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Unhealthy

Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-8d451c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-8d451c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-8d451c-7d758d8fdf from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-8d451c-7d758d8fdf

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-8d451c-7d758d8fdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-8d451c: serviceaccount "payload-processing-e2e-explicit-8d451c" not found

openshift-ingress

multus

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

Created

Created container: payload-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-8d451c: serviceaccount "payload-processing-e2e-explicit-8d451c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

openshift-ingress

kubelet

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-8d451c-7d758d8fdf

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr
(x6)

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-39ea7c6f-69f5574754-lmcks

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-39ea7c6f-fd968b857-7mq2k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-39ea7c6f-openshift-default-566bd9c7d4-j2v5j

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

ProbeError

Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

Unhealthy

Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d

SuccessfulCreate

Created pod: e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

openshift-ingress

deployment-controller

e2e-mt-7b8e72c7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-7b8e72c7-openshift-default-69b8bddc8d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-7b8e72c7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-7b8e72c7

openshift-ingress

multus

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7b8e72c7: serviceaccount "payload-processing-e2e-mt-7b8e72c7" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-7b8e72c7-66f454f4bf

SuccessfulCreate

Created pod: payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-7b8e72c7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-7b8e72c7-66f454f4bf

FailedCreate

Error creating: pods "payload-processing-e2e-mt-7b8e72c7-66f454f4bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7b8e72c7: serviceaccount "payload-processing-e2e-mt-7b8e72c7" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-7b8e72c7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-7b8e72c7-66f454f4bf from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-8d451c-7d7b4b9f74-xw47v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-8d451c-7d758d8fdf-hdnqr

Unhealthy

Readiness probe failed: dial tcp 10.134.0.62:9004: connect: connection refused

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

ProbeError

Readiness probe error: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Unhealthy

Readiness probe failed: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b8e72c7-7df96bbb58-cjwl7

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-7b8e72c7

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-7b8e72c7: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-7b8e72c7": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-7b8e72c7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 60e037e4-ce36-4412-8a4b-c5a38f816f5f, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-mt-7b8e72c7-66f454f4bf-k6l45

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-7b8e72c7-openshift-default-69b8bddc8d-9t2bk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

e2e-shared-a-w2-799002-openshift-default-69d7698d8d

SuccessfulCreate

Created pod: e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

openshift-ingress

deployment-controller

e2e-shared-a-w2-799002-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-799002-openshift-default-69d7698d8d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-799002

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-799002

openshift-ingress

multus

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-799002

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-799002-d567ff65f from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-799002: serviceaccount "payload-processing-e2e-shared-a-w2-799002" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-799002-d567ff65f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-799002-d567ff65f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-799002: serviceaccount "payload-processing-e2e-shared-a-w2-799002" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-799002-d567ff65f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-799002

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878

SuccessfulCreate

Created pod: e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-71ba6d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-71ba6d

openshift-ingress

deployment-controller

e2e-shared-b-w2-71ba6d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878 from 0 to 1

openshift-ingress

multus

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-explicit-8d451c-openshift-default-7b5868fcdd-g9rzj

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-71ba6d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-71ba6d: serviceaccount "payload-processing-e2e-shared-b-w2-71ba6d" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-71ba6d: serviceaccount "payload-processing-e2e-shared-b-w2-71ba6d" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-71ba6d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-noten-836d37

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-836d37

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

e2e-noten-836d37-openshift-default-58bc769474

SuccessfulCreate

Created pod: e2e-noten-836d37-openshift-default-58bc769474-4gtk6

openshift-ingress

deployment-controller

e2e-noten-836d37-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-836d37-openshift-default-58bc769474 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp

Created

Created container: payload-processing

openshift-ingress

multus

e2e-noten-836d37-openshift-default-58bc769474-4gtk6

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-836d37-openshift-default-58bc769474-4gtk6

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-noten-836d37-openshift-default-58bc769474-4gtk6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-836d37-openshift-default-58bc769474-4gtk6

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-cef92e

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-cef92e: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-cef92e": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-cef92e, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8c0b8fde-1fdc-4098-8d4b-6adf89dd663e, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-cef92e-5678c5544b-4jp9d

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-cef92e-65546b8d85-v27x7

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-mt-194eb64c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-194eb64c

openshift-ingress

deployment-controller

e2e-mt-194eb64c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-194eb64c-openshift-default-779f4656cb from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-194eb64c-openshift-default-779f4656cb

SuccessfulCreate

Created pod: e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

openshift-ingress

multus

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9

FailedCreate

Error creating: pods "payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-194eb64c: serviceaccount "payload-processing-e2e-mt-194eb64c" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9

SuccessfulCreate

Created pod: payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-194eb64c: serviceaccount "payload-processing-e2e-mt-194eb64c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-194eb64c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-194eb64c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-194eb64c-5b7bb5f7d9 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-a-cef92e-openshift-default-685dfdf9fd-knbjc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-6ba087-bdcc845c4-cqzxs

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-6ba087-7f7fcd6f95-j2vlf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-891e67-fc5bccbf4-4w5pk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-891e67-55d8c8cf55-j7jmn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w0-6ba087-openshift-default-85464b8f68-b5knz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-noten-836d37-openshift-default-58bc769474-4gtk6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-b-891e67-openshift-default-564fcdf9fb-xw8kt

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-2482ac

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-2482ac

openshift-ingress

deployment-controller

e2e-shared-a-w3-2482ac-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8

SuccessfulCreate

Created pod: e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

openshift-ingress

multus

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-2482ac

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-2482ac

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-2482ac: serviceaccount "payload-processing-e2e-shared-a-w3-2482ac" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-2482ac: serviceaccount "payload-processing-e2e-shared-a-w3-2482ac" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-9433b9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-9433b9

openshift-ingress

replicaset-controller

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74

SuccessfulCreate

Created pod: e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

openshift-ingress

deployment-controller

e2e-shared-b-w3-9433b9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-9433b9-openshift-default-5754d85b74 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Started

Started container istio-proxy

openshift-ingress

multus

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-9433b9: serviceaccount "payload-processing-e2e-shared-b-w3-9433b9" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-9433b9: serviceaccount "payload-processing-e2e-shared-b-w3-9433b9" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-9433b9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-9433b9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-9433b9-6746686dd5 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

Started

Started container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

ProbeError

Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Unhealthy

Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

ProbeError

Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Unhealthy

Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-mt-194eb64c-5b7bb5f7d9-v9knf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-194eb64c-7f5bc6c478-fhdj6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-194eb64c-openshift-default-779f4656cb-lcpcg

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-api-a-f1f91d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-f1f91d

openshift-ingress

deployment-controller

e2e-api-a-f1f91d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-f1f91d-openshift-default-7b546d6c86 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-f1f91d-openshift-default-7b546d6c86

SuccessfulCreate

Created pod: e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

openshift-ingress

multus

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

AddedInterface

Add eth0 [10.132.0.83/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-f1f91d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-f1f91d-89966b557

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-f1f91d-89966b557-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f1f91d: serviceaccount "payload-processing-e2e-api-a-f1f91d" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f1f91d: serviceaccount "payload-processing-e2e-api-a-f1f91d" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-f1f91d-89966b557

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-f1f91d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-f1f91d-89966b557 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

AddedInterface

Add eth0 [10.134.0.81/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

e2e-api-b-816605-openshift-default-76dd9c445d

SuccessfulCreate

Created pod: e2e-api-b-816605-openshift-default-76dd9c445d-259dd

openshift-ingress

multus

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-api-b-816605-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-816605-openshift-default-76dd9c445d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-816605

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-816605
(x5)

openshift-ingress

kubelet

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

Unhealthy

Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

ProbeError

Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-816605-85b8cdc57c

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-816605

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-816605-7d6f684b86 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-816605-7d6f684b86

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-816605-85b8cdc57c

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-816605-85b8cdc57c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-816605: serviceaccount "payload-processing-e2e-api-b-816605" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-816605-7d6f684b86

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-816605-7d6f684b86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-816605: serviceaccount "payload-processing-e2e-api-b-816605" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-816605

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-816605-85b8cdc57c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6

AddedInterface

Add eth0 [10.134.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-651569-5b8f855699-ss9dk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-651569-57fcd4cddb-cfmdp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w4-651569-openshift-default-cdcc9bdb4-2hsjf

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Unhealthy

Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

ProbeError

Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-f1f91d-89966b557-gzxnx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f1f91d-75ffbb7fd-sj4db

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-a-f1f91d-openshift-default-7b546d6c86-kmhkm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-816605-7d6f684b86-h22p6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-816605-85b8cdc57c-2jjg5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-b-816605-openshift-default-76dd9c445d-259dd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-799002-d567ff65f-z74b7

Unhealthy

Readiness probe failed: dial tcp 10.132.0.68:9004: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-799002-b8d69fd78-5chn7

Unhealthy

Readiness probe failed: dial tcp 10.132.0.67:9004: connect: connection refused

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-a-w2-799002

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-w2-799002: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-w2-799002": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-w2-799002, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: d3cf3a33-979b-482e-950c-e7d33f98d462, UID in object meta:

openshift-ingress

kubelet

e2e-shared-a-w2-799002-openshift-default-69d7698d8d-gfrd4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-71ba6d-6fd54f89c8-dgxjl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-71ba6d-5dbcff57-jr8mp

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w2-71ba6d

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w2-71ba6d: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w2-71ba6d": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w2-71ba6d, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 1b510fd5-76e4-44c0-906b-ecfa1c7fa2b0, UID in object meta:

openshift-ingress

kubelet

e2e-shared-b-w2-71ba6d-openshift-default-765f5dc878-82nr2

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ipp-a-976998-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-976998-openshift-default-c8888dc98 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-976998-openshift-default-c8888dc98

SuccessfulCreate

Created pod: e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-976998

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-976998

openshift-ingress

multus

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

AddedInterface

Add eth0 [10.134.0.88/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

Created

Created container: payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-976998

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-976998-758f86d9c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-976998

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-976998-758f86d9c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-976998: serviceaccount "payload-processing-e2e-ipp-a-976998" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

AddedInterface

Add eth0 [10.134.0.89/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-976998-758f86d9c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-976998-758f86d9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-976998: serviceaccount "payload-processing-e2e-ipp-a-976998" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

AddedInterface

Add eth0 [10.134.0.90/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Unhealthy

Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": read tcp 10.134.0.2:42142->10.134.0.88:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

ProbeError

Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": read tcp 10.134.0.2:42142->10.134.0.88:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

ProbeError

Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Unhealthy

Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6

SuccessfulCreate

Created pod: e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

openshift-ingress

deployment-controller

e2e-ipp-b-bae8d3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-bae8d3-openshift-default-6c47f765b6 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-bae8d3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-bae8d3

openshift-ingress

multus

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

AddedInterface

Add eth0 [10.134.0.91/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-bae8d3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

AddedInterface

Add eth0 [10.132.0.90/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bae8d3: serviceaccount "payload-processing-e2e-ipp-b-bae8d3" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

AddedInterface

Add eth0 [10.132.0.91/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bae8d3: serviceaccount "payload-processing-e2e-ipp-b-bae8d3" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-bae8d3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4 from 0 to 1

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Unhealthy

Readiness probe failed: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

ProbeError

Readiness probe error: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

ProbeError

Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Unhealthy

Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-ipp-cleanup-40e6bb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d

SuccessfulCreate

Created pod: e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-40e6bb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-40e6bb

openshift-ingress

multus

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-40e6bb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-40e6bb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-40e6bb: serviceaccount "payload-processing-e2e-ipp-cleanup-40e6bb" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-40e6bb: serviceaccount "payload-processing-e2e-ipp-cleanup-40e6bb" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Unhealthy

Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

ProbeError

Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-40e6bb-698cf8bff8-hqtvs

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-40e6bb-688df5f6c7-5srks

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-cleanup-40e6bb-openshift-default-579dfcb59d-h7rck

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-976998-fd6c96c5d-r9btc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-976998-758f86d9c-c6lsl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-a-976998-openshift-default-c8888dc98-lw4rm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bae8d3-5db4c6fdd5-ldnjc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bae8d3-7ddfd8cc4-tmb2s

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-bae8d3-openshift-default-6c47f765b6-tt4d2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2482ac-6c4ddccb95-xs7tc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2482ac-75f7b999d6-bjvlg

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-a-w3-2482ac

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2482ac: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-w3-2482ac": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2482ac, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: de3d26c1-d8f4-4f0f-8530-1e33f3e8cc81, UID in object meta:

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-a-w3-2482ac-openshift-default-75b5d5c9b8-zfr9t

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-9433b9-6746686dd5-5b2nd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-9433b9-6b64d44946-bt9h7

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w3-9433b9

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9433b9: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w3-9433b9": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w3-9433b9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: eece46da-2ed1-42f7-aeef-248eee3d1810, UID in object meta:

openshift-ingress

kubelet

e2e-shared-b-w3-9433b9-openshift-default-5754d85b74-mvnvv

Killing

Stopping container istio-proxy