Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Scheduled

Successfully assigned openshift-ingress/e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

Scheduled

Successfully assigned openshift-ingress/e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j to ip-10-0-138-138.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

Scheduled

Successfully assigned openshift-ingress/e2e-noten-8c963b-openshift-default-6888687474-pv7hb to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8 to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

Scheduled

Successfully assigned openshift-ingress/e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4 to ip-10-0-137-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2 to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

Scheduled

Successfully assigned openshift-ingress/e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-b06908-64c748b995-9qfh6 to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-4faba0-openshift-default-55b947b884-8h457 to ip-10-0-142-105.ec2.internal

openshift-ingress

kube-auth-proxy-7bc8cbff6f-w2rcw

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-7bc8cbff6f-w2rcw to ip-10-0-137-104.ec2.internal

openshift-ingress

kube-auth-proxy-7bc8cbff6f-6vkds

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-7bc8cbff6f-6vkds to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22 to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh to ip-10-0-138-138.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-57dw5

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-57dw5 to ip-10-0-137-104.ec2.internal

openshift-ingress

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml to ip-10-0-137-104.ec2.internal

openshift-ingress

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Scheduled

Successfully assigned openshift-ingress/e2e-mt-d245d137-openshift-default-c948444cb-ggjfw to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm to ip-10-0-137-104.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-cmpkf

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-cmpkf to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48 to ip-10-0-138-138.ec2.internal

openshift-ingress

router-default-64665745fb-n4rxl

Scheduled

Successfully assigned openshift-ingress/router-default-64665745fb-n4rxl to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4 to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-8k6rh

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-8k6rh to ip-10-0-137-104.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-kkjdk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-kkjdk to ip-10-0-137-104.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2 to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Scheduled

Successfully assigned openshift-ingress/e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l to ip-10-0-142-105.ec2.internal

openshift-ingress

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87 to ip-10-0-138-138.ec2.internal

openshift-ingress

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2 to ip-10-0-142-105.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-6v2jt to ip-10-0-137-104.ec2.internal

openshift-ingress

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-b06908-openshift-default-bd57dd984-6wddb to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh to ip-10-0-142-105.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7 to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv to ip-10-0-138-138.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv to ip-10-0-138-138.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-64665745fb from 0 to 1
(x8)

openshift-ingress

replicaset-controller

router-default-64665745fb

FailedCreate

Error creating: pods "router-default-64665745fb-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-64665745fb

FailedCreate

Error creating: pods "router-default-64665745fb-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-64665745fb

SuccessfulCreate

Created pod: router-default-64665745fb-n4rxl
(x6)

openshift-ingress

kubelet

router-default-64665745fb-n4rxl

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x6)

openshift-ingress

kubelet

router-default-64665745fb-n4rxl

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-64665745fb-n4rxl

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eaa467bc105be2f7695ecfea0f0c53f4c5cd52f1f705da378c7a33d13f1d102f" already present on machine

openshift-ingress

multus

router-default-64665745fb-n4rxl

AddedInterface

Add eth0 [10.133.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-64665745fb-n4rxl

Started

Started container router

openshift-ingress

kubelet

router-default-64665745fb-n4rxl

Created

Created container: router

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-w2rcw

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

kube-auth-proxy-7bc8cbff6f

SuccessfulCreate

Created pod: kube-auth-proxy-7bc8cbff6f-w2rcw

openshift-ingress

multus

kube-auth-proxy-7bc8cbff6f-w2rcw

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

kube-auth-proxy-7bc8cbff6f

SuccessfulCreate

Created pod: kube-auth-proxy-7bc8cbff6f-6vkds

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-7bc8cbff6f from 0 to 2
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-7bc8cbff6f

FailedCreate

Error creating: pods "kube-auth-proxy-7bc8cbff6f-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-6vkds

FailedMount

MountVolume.SetUp failed for volume "kube-api-access-msjlg" : failed to sync configmap cache: timed out waiting for the condition

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

kube-auth-proxy-7bc8cbff6f-6vkds

AddedInterface

Add eth0 [10.132.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-6vkds

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-57dw5

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-57dw5

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-w2rcw

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-w2rcw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 3.047s (3.047s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-w2rcw

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-6vkds

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.456s (2.456s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-6vkds

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7bc8cbff6f-6vkds

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.248s (2.248s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.851s (2.851s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-86ljw

Started

Started container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-cmpkf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-cmpkf

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-cmpkf

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-cmpkf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.334s (2.334s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-cmpkf

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-cmpkf

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-cmpkf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-cmpkf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-57dw5

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-57dw5

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-6v2jt

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.885s (2.885s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1

openshift-ingress

multus

payload-processing-5b54b7fbb7-8k6rh

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-kkjdk

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-kkjdk
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-8k6rh

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-8k6rh

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kkjdk

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kkjdk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-8k6rh

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.24s (2.24s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kkjdk

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-8k6rh

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-8k6rh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kkjdk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.272s (2.272s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Unhealthy

Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

ProbeError

Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Created

Created container: istio-proxy
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-6v2jt

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-inf-a-c31adf-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-c31adf-openshift-default-8597b8b85d from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w0-5d8053-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-5d8053-openshift-default-9b5dcddf8 from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w4-b18433-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-b18433-openshift-default-7588884dfd from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-b18433-openshift-default-7588884dfd

SuccessfulCreate

Created pod: e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-b18433

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-b18433

openshift-ingress

replicaset-controller

e2e-inf-a-c31adf-openshift-default-8597b8b85d

SuccessfulCreate

Created pod: e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

openshift-ingress

gateway_labeler_controller

e2e-inf-a-c31adf

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c31adf

openshift-ingress

replicaset-controller

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8

SuccessfulCreate

Created pod: e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-5d8053

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-5d8053

openshift-ingress

multus

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

multus

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c31adf: serviceaccount "payload-processing-e2e-inf-a-c31adf" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-c31adf

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-c31adf

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-c31adf-5455b445cc from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-c31adf-5455b445cc

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-c31adf-5455b445cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c31adf: serviceaccount "payload-processing-e2e-inf-a-c31adf" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-c31adf-5455b445cc

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-b18433

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-b18433-69b8b6ff7c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-b18433

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-b18433: serviceaccount "payload-processing-e2e-worker-w4-b18433" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-b18433: serviceaccount "payload-processing-e2e-worker-w4-b18433" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-5d8053: serviceaccount "payload-processing-e2e-worker-w0-5d8053" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-5d8053

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-5d8053

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-5d8053: serviceaccount "payload-processing-e2e-worker-w0-5d8053" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.529s (1.529s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 423ms (423ms including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 415ms (415ms including waiting). Image size: 166449312 bytes.

openshift-ingress

deployment-controller

e2e-ait-015dd779-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-015dd779-openshift-default-67789f49cd from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-015dd779-openshift-default-67789f49cd

SuccessfulCreate

Created pod: e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

openshift-ingress

gateway_labeler_controller

e2e-ait-015dd779

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-015dd779

openshift-ingress

replicaset-controller

e2e-autores-b200d5-openshift-default-595dcb7776

SuccessfulCreate

Created pod: e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

openshift-ingress

multus

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-autores-b200d5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-b200d5-openshift-default-595dcb7776 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-autores-b200d5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-b200d5

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.925s (2.925s including waiting). Image size: 181318953 bytes.

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-015dd779: serviceaccount "payload-processing-e2e-ait-015dd779" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-015dd779

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-015dd779-7857cdcf7d from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-015dd779-5886fb7969

FailedCreate

Error creating: pods "payload-processing-e2e-ait-015dd779-5886fb7969-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-015dd779: serviceaccount "payload-processing-e2e-ait-015dd779" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-015dd779-5886fb7969

SuccessfulCreate

Created pod: payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

openshift-ingress

deployment-controller

payload-processing-e2e-ait-015dd779

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-015dd779-5886fb7969 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.199s (1.199s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.219s (1.219s including waiting). Image size: 166449312 bytes.

openshift-ingress

GatewayAuthBootstrap

e2e-inf-b-99ac10

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-inf-b-99ac10-authn-ssl: envoyfilters.networking.istio.io "e2e-inf-b-99ac10-authn-ssl" already exists

openshift-ingress

gateway_labeler_controller

e2e-inf-b-99ac10

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-99ac10

openshift-ingress

deployment-controller

e2e-inf-b-99ac10-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-99ac10-openshift-default-cdc8788b6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-b-99ac10-openshift-default-cdc8788b6

SuccessfulCreate

Created pod: e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

openshift-ingress

multus

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-autores-b200d5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-b200d5-d84d9c6f9 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-b200d5-d84d9c6f9

SuccessfulCreate

Created pod: payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-b200d5-65cd94894b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-b200d5-d84d9c6f9

FailedCreate

Error creating: pods "payload-processing-e2e-autores-b200d5-d84d9c6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-b200d5: serviceaccount "payload-processing-e2e-autores-b200d5" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-b200d5-65cd94894b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-b200d5-65cd94894b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-b200d5: serviceaccount "payload-processing-e2e-autores-b200d5" not found

openshift-ingress

multus

payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-b200d5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-b200d5-65cd94894b from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-99ac10

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

Started

Started container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-99ac10

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-99ac10-76ccc4dc57 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-99ac10-76ccc4dc57

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-99ac10-76ccc4dc57

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-99ac10-76ccc4dc57-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-99ac10: serviceaccount "payload-processing-e2e-inf-b-99ac10" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-99ac10: serviceaccount "payload-processing-e2e-inf-b-99ac10" not found
(x2)

openshift-ingress

kubelet

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

Created

Created container: istio-proxy

openshift-ingress

multus

payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

Unhealthy

Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

ProbeError

Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Created

Created container: istio-proxy
(x6)

openshift-ingress

kubelet

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

ProbeError

Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

Unhealthy

Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ait-015dd779-5886fb7969-hwgdd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-015dd779-7857cdcf7d-88d5n

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-015dd779-openshift-default-67789f49cd-gzkj9

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d

SuccessfulCreate

Created pod: e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

openshift-ingress

deployment-controller

e2e-migrate-4390f8ca-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

ProbeError

Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Unhealthy

Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-migrate-4390f8ca-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-4390f8ca-gw
(x5)

openshift-ingress

kubelet

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

Unhealthy

Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

ProbeError

Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-4390f8ca

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-4390f8ca

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-4390f8ca-75685fb6db from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-4390f8ca-75685fb6db

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-4390f8ca: serviceaccount "payload-processing-e2e-migrate-4390f8ca" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-4390f8ca-75685fb6db

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-4390f8ca-75685fb6db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-4390f8ca: serviceaccount "payload-processing-e2e-migrate-4390f8ca" not found

openshift-ingress

kubelet

payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

Unhealthy

Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

ProbeError

Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-migrate-4390f8ca-75685fb6db-f6xrg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-migrate-4390f8ca-gw-openshift-default-768c8f688d-bf9pv

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-4390f8ca-757c8c749d-q8s2g

Killing

Stopping container payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-7fd35bf2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-7fd35bf2

openshift-ingress

deployment-controller

e2e-ait-7fd35bf2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-7fd35bf2-openshift-default-57b5ffffd6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6

SuccessfulCreate

Created pod: e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

openshift-ingress

multus

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-7fd35bf2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7fd35bf2: serviceaccount "payload-processing-e2e-ait-7fd35bf2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-7fd35bf2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-7fd35bf2-8887d647c from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-7fd35bf2-8887d647c

FailedCreate

Error creating: pods "payload-processing-e2e-ait-7fd35bf2-8887d647c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7fd35bf2: serviceaccount "payload-processing-e2e-ait-7fd35bf2" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-7fd35bf2-8887d647c

SuccessfulCreate

Created pod: payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

Unhealthy

Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

ProbeError

Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7fd35bf2-76b76bdb6b-m62ms

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-7fd35bf2-8887d647c-zhrbv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-7fd35bf2-openshift-default-57b5ffffd6-jbnms

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-derive-700d65b8-openshift-default-5cc8595d64

SuccessfulCreate

Created pod: e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

openshift-ingress

gateway_labeler_controller

e2e-derive-700d65b8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-700d65b8

openshift-ingress

deployment-controller

e2e-derive-700d65b8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-700d65b8-openshift-default-5cc8595d64 from 0 to 1

openshift-ingress

multus

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-700d65b8-555cd59c5c

FailedCreate

Error creating: pods "payload-processing-e2e-derive-700d65b8-555cd59c5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-700d65b8: serviceaccount "payload-processing-e2e-derive-700d65b8" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-700d65b8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-700d65b8-78cd6865f4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-derive-700d65b8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-700d65b8-555cd59c5c from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-700d65b8: serviceaccount "payload-processing-e2e-derive-700d65b8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-700d65b8-555cd59c5c

SuccessfulCreate

Created pod: payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-autores-b200d5-d84d9c6f9-n55r8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b200d5-65cd94894b-vbcx2

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-700d65b8-78cd6865f4-jhnjv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-700d65b8-555cd59c5c-hg98d

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-derive-700d65b8-openshift-default-5cc8595d64-l64jk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

e2e-explicit-83507f-openshift-default-778bdbb6d8

SuccessfulCreate

Created pod: e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

openshift-ingress

gateway_labeler_controller

e2e-explicit-83507f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-83507f

openshift-ingress

deployment-controller

e2e-explicit-83507f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-83507f-openshift-default-778bdbb6d8 from 0 to 1

openshift-ingress

kubelet

e2e-autores-b200d5-openshift-default-595dcb7776-j5c7v

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-83507f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-83507f-6987f5558f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-83507f-6987f5558f

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-83507f-6987f5558f

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-83507f-6987f5558f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-83507f: serviceaccount "payload-processing-e2e-explicit-83507f" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-83507f-96bbc5b66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-83507f-96bbc5b66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-83507f: serviceaccount "payload-processing-e2e-explicit-83507f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-83507f-96bbc5b66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-83507f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-83507f-96bbc5b66 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Unhealthy

Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

ProbeError

Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-mt-d62f3adc-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-d62f3adc-openshift-default-89d5c776d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-d62f3adc

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d62f3adc

openshift-ingress

replicaset-controller

e2e-mt-d62f3adc-openshift-default-89d5c776d

SuccessfulCreate

Created pod: e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

openshift-ingress

multus

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-d62f3adc

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-d62f3adc-546fbd775c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-d62f3adc

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d62f3adc: serviceaccount "payload-processing-e2e-mt-d62f3adc" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d62f3adc-546fbd775c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-d62f3adc-546fbd775c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d62f3adc: serviceaccount "payload-processing-e2e-mt-d62f3adc" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d62f3adc-546fbd775c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-explicit-83507f-6987f5558f-ncp7x

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-83507f-96bbc5b66-rmqsv

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

Unhealthy

Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

ProbeError

Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-noten-8c963b-openshift-default-6888687474

SuccessfulCreate

Created pod: e2e-noten-8c963b-openshift-default-6888687474-pv7hb

openshift-ingress

deployment-controller

e2e-noten-8c963b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-8c963b-openshift-default-6888687474 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-noten-8c963b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-8c963b

openshift-ingress

kubelet

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

Created

Created container: istio-proxy

openshift-ingress

multus

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-explicit-83507f-openshift-default-778bdbb6d8-msg22

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d62f3adc-65b6cff78b-8n72w

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d62f3adc-546fbd775c-9msdh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-d62f3adc-openshift-default-89d5c776d-rjw4j

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-a-w2-1a9245-openshift-default-78b847994

SuccessfulCreate

Created pod: e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-1a9245

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-1a9245

openshift-ingress

deployment-controller

e2e-shared-a-w2-1a9245-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-1a9245-openshift-default-78b847994 from 0 to 1

openshift-ingress

multus

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-1a9245

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-1a9245: serviceaccount "payload-processing-e2e-shared-a-w2-1a9245" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-1a9245: serviceaccount "payload-processing-e2e-shared-a-w2-1a9245" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-1a9245

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4

SuccessfulCreate

Created pod: e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-03270e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-03270e

openshift-ingress

deployment-controller

e2e-shared-b-w2-03270e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4 from 0 to 1

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Unhealthy

Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

ProbeError

Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Started

Started container istio-proxy

openshift-ingress

multus

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-03270e: serviceaccount "payload-processing-e2e-shared-b-w2-03270e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-03270e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-03270e: serviceaccount "payload-processing-e2e-shared-b-w2-03270e" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-03270e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-8c963b-openshift-default-6888687474-pv7hb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-inf-a-c31adf-5455b445cc-5mmml

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-c31adf

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-c31adf: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-c31adf": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-c31adf, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 91a4a89e-2bb8-481b-bedf-bf772f42ead9, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-c31adf-7d667f4f4-hdxk4

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-mt-d245d137-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-d245d137-openshift-default-c948444cb from 0 to 1

openshift-ingress

multus

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-mt-d245d137

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d245d137

openshift-ingress

replicaset-controller

e2e-mt-d245d137-openshift-default-c948444cb

SuccessfulCreate

Created pod: e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-a-c31adf-openshift-default-8597b8b85d-6g7pj

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-processing-e2e-mt-d245d137

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-d245d137-5c4c7b86d5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d245d137-856dcdb558

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

openshift-ingress

multus

payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d245d137-856dcdb558

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-d245d137-856dcdb558-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d245d137: serviceaccount "payload-processing-e2e-mt-d245d137" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

Created

Created container: payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-d245d137

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-d245d137-856dcdb558 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d245d137-5c4c7b86d5

FailedCreate

Error creating: pods "payload-processing-e2e-mt-d245d137-5c4c7b86d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d245d137: serviceaccount "payload-processing-e2e-mt-d245d137" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d245d137-5c4c7b86d5

SuccessfulCreate

Created pod: payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-99ac10-67f9d6f7fd-7p7dr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-99ac10-76ccc4dc57-ggxcd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

ProbeError

Readiness probe error: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Unhealthy

Readiness probe failed: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-99ac10-openshift-default-cdc8788b6-fcvwh

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-shared-a-w3-95a92b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8

SuccessfulCreate

Created pod: e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-95a92b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-95a92b

openshift-ingress

multus

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-95a92b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-95a92b: serviceaccount "payload-processing-e2e-shared-a-w3-95a92b" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-95a92b: serviceaccount "payload-processing-e2e-shared-a-w3-95a92b" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-95a92b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

AddedInterface

Add eth0 [10.133.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d

SuccessfulCreate

Created pod: e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-d81d7c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-d81d7c

openshift-ingress

deployment-controller

e2e-shared-b-w3-d81d7c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d from 0 to 1

openshift-ingress

multus

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-d81d7c-56db445699

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-d81d7c-56db445699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d81d7c: serviceaccount "payload-processing-e2e-shared-b-w3-d81d7c" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-d81d7c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-d81d7c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-d81d7c-56db445699 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d81d7c: serviceaccount "payload-processing-e2e-shared-b-w3-d81d7c" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-d81d7c-56db445699

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

ProbeError

Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Unhealthy

Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x6)

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d245d137-856dcdb558-gm9tq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d245d137-5c4c7b86d5-kjpb2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-d245d137-openshift-default-c948444cb-ggjfw

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-api-a-4faba0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-4faba0

openshift-ingress

deployment-controller

e2e-api-a-4faba0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-4faba0-openshift-default-55b947b884 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-4faba0-openshift-default-55b947b884

SuccessfulCreate

Created pod: e2e-api-a-4faba0-openshift-default-55b947b884-8h457

openshift-ingress

multus

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-4faba0: serviceaccount "payload-processing-e2e-api-a-4faba0" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-4faba0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-4faba0-9f944b5ff

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-4faba0-9f944b5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-4faba0: serviceaccount "payload-processing-e2e-api-a-4faba0" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-4faba0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-4faba0-9f944b5ff from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-4faba0-9f944b5ff

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

openshift-ingress

multus

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

e2e-api-b-b06908-openshift-default-bd57dd984

SuccessfulCreate

Created pod: e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

openshift-ingress

gateway_labeler_controller

e2e-api-b-b06908

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-b06908

openshift-ingress

deployment-controller

e2e-api-b-b06908-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-b06908-openshift-default-bd57dd984 from 0 to 1

openshift-ingress

multus

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-b06908

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-b06908-64c748b995 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b06908: serviceaccount "payload-processing-e2e-api-b-b06908" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-b06908-64c748b995

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

openshift-ingress

multus

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-b06908

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-b06908-7dd968cb9d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx

Created

Created container: payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-b06908-64c748b995

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-b06908-64c748b995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-b06908: serviceaccount "payload-processing-e2e-api-b-b06908" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx
(x2)

openshift-ingress

kubelet

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

ProbeError

Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

Unhealthy

Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-b18433-5d46fd8dbb-p8ws8

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-worker-w4-b18433

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-worker-w4-b18433: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-worker-w4-b18433": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-worker-w4-b18433, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a21c1c9f-64e0-487b-9477-ace808a8bbd6, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-b18433-69b8b6ff7c-fkzwm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-4faba0-9f944b5ff-swnmv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-4faba0-78d9bc56d6-7msrb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w4-b18433-openshift-default-7588884dfd-pfcv9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

Unhealthy

Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-api-a-4faba0-openshift-default-55b947b884-8h457

ProbeError

Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-b06908-7dd968cb9d-8cglx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-b06908-64c748b995-9qfh6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-b06908-openshift-default-bd57dd984-6wddb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-5d8053-6b4b7f7cbf-tmw48

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-5d8053-74f9f9cf86-k464g

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-1a9245-57c455d64b-f56tt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-1a9245-5f57cc69c-kdgk7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w0-5d8053-openshift-default-9b5dcddf8-qm6dz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w2-1a9245-openshift-default-78b847994-nl847

Killing

Stopping container istio-proxy

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w2-03270e

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w2-03270e: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w2-03270e": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w2-03270e, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 417a9175-499b-4f8a-922a-10be8cb03fd5, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-03270e-5ff5c9d4bf-ccsdv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-03270e-76dcc585c4-lkg87

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w2-03270e-openshift-default-6bd7454cc4-rqwq4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-f72155

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f72155

openshift-ingress

deployment-controller

e2e-ipp-a-f72155-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-f72155-openshift-default-6fdd7dc55d from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d

SuccessfulCreate

Created pod: e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

openshift-ingress

multus

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

AddedInterface

Add eth0 [10.133.0.83/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-f72155-b6c84c8fc

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-f72155-b6c84c8fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f72155: serviceaccount "payload-processing-e2e-ipp-a-f72155" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f72155: serviceaccount "payload-processing-e2e-ipp-a-f72155" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-f72155-b6c84c8fc

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-f72155

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-f72155-b6c84c8fc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-f72155

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7

AddedInterface

Add eth0 [10.133.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

Created

Created container: payload-processing

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Unhealthy

Readiness probe failed: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

ProbeError

Readiness probe error: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Unhealthy

Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

ProbeError

Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6

SuccessfulCreate

Created pod: e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

openshift-ingress

deployment-controller

e2e-ipp-b-0f4423-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-0f4423-openshift-default-5cd99f98b6 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-0f4423

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-0f4423

openshift-ingress

multus

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

AddedInterface

Add eth0 [10.133.0.85/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-0f4423-678745d849

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-0f4423-678745d849-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0f4423: serviceaccount "payload-processing-e2e-ipp-b-0f4423" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-0f4423

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-0f4423-678745d849 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-0f4423

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-0f4423-678745d849

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0f4423: serviceaccount "payload-processing-e2e-ipp-b-0f4423" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

AddedInterface

Add eth0 [10.133.0.86/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2

AddedInterface

Add eth0 [10.132.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Unhealthy

Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

ProbeError

Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Unhealthy

Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

ProbeError

Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-c37358

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-c37358

openshift-ingress

deployment-controller

e2e-ipp-cleanup-c37358-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-c37358-openshift-default-6d947fd594 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594

SuccessfulCreate

Created pod: e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

openshift-ingress

multus

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

AddedInterface

Add eth0 [10.132.0.87/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-c37358: serviceaccount "payload-processing-e2e-ipp-cleanup-c37358" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-c37358

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-c37358: serviceaccount "payload-processing-e2e-ipp-cleanup-c37358" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

AddedInterface

Add eth0 [10.133.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

AddedInterface

Add eth0 [10.132.0.88/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-c37358

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

ProbeError

Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

Unhealthy

Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c37358-79fdf68f97-5cph2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c37358-59bd4c4554-wgh7s

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-cleanup-c37358

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c37358: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-cleanup-c37358": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c37358, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 530007cf-b88a-49c9-b6b3-55afdf5c1a6f, UID in object meta:

openshift-ingress

kubelet

e2e-ipp-cleanup-c37358-openshift-default-6d947fd594-6tw7g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-f72155-8c78f8c78-68rm7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-f72155-b6c84c8fc-ws9vf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-f72155-openshift-default-6fdd7dc55d-ftz5s

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0f4423-6cbf66b598-47v45

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0f4423-678745d849-qlnl2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-0f4423-openshift-default-5cd99f98b6-fdbsf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-95a92b-78db98d6d7-x4xkc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-95a92b-6c876cbd66-rpwcm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-a-w3-95a92b-openshift-default-5dbf44ffb8-vq62d

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-d81d7c-56db445699-n7cxv

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w3-d81d7c

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d81d7c: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w3-d81d7c": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d81d7c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 415c3d9a-6ae3-4a2a-9698-1fe946944a42, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-d81d7c-865596897d-tnr2l

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w3-d81d7c-openshift-default-66d75f9c4d-jj59s

Killing

Stopping container istio-proxy