Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-45sdg

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-45sdg to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

Scheduled

Successfully assigned openshift-ingress/e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db to ip-10-0-129-252.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-mv7q9

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-mv7q9 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5 to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8 to ip-10-0-129-227.ec2.internal

openshift-ingress

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp to ip-10-0-129-227.ec2.internal

openshift-ingress

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99 to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Scheduled

Successfully assigned openshift-ingress/e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz to ip-10-0-129-252.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-868tl

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-868tl to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64 to ip-10-0-129-227.ec2.internal

openshift-ingress

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9 to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m to ip-10-0-129-252.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ht96l to ip-10-0-129-227.ec2.internal

openshift-ingress

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-558d9b8847-2lchj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-558d9b8847-2lchj to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Scheduled

Successfully assigned openshift-ingress/e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-74969cc5f8-tflk8

Scheduled

Successfully assigned openshift-ingress/payload-processing-74969cc5f8-tflk8 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-910e11-996975df7-287b8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-910e11-996975df7-287b8 to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-cldpb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-cldpb to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

Scheduled

Successfully assigned openshift-ingress/e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88 to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh to ip-10-0-129-227.ec2.internal

openshift-ingress

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7 to ip-10-0-129-227.ec2.internal

openshift-ingress

router-default-79988fc9d-2hkvw

Scheduled

Successfully assigned openshift-ingress/router-default-79988fc9d-2hkvw to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7 to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss to ip-10-0-129-227.ec2.internal

openshift-ingress

kube-auth-proxy-5b76b67bf4-xtqs9

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5b76b67bf4-xtqs9 to ip-10-0-132-28.ec2.internal

openshift-ingress

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

Scheduled

Successfully assigned openshift-ingress/e2e-derive-b6741af4-openshift-default-f4c9fb88-796br to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6 to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7 to ip-10-0-129-227.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk to ip-10-0-129-252.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp to ip-10-0-129-252.ec2.internal

openshift-ingress

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5 to ip-10-0-129-252.ec2.internal

openshift-ingress

kube-auth-proxy-5b76b67bf4-tsmb8

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5b76b67bf4-tsmb8 to ip-10-0-129-227.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf to ip-10-0-129-252.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-79988fc9d from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x14)

openshift-ingress

replicaset-controller

router-default-79988fc9d

FailedCreate

Error creating: pods "router-default-79988fc9d-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-79988fc9d

FailedCreate

Error creating: pods "router-default-79988fc9d-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-79988fc9d

SuccessfulCreate

Created pod: router-default-79988fc9d-2hkvw
(x6)

openshift-ingress

kubelet

router-default-79988fc9d-2hkvw

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-79988fc9d-2hkvw

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-79988fc9d-2hkvw

AddedInterface

Add eth0 [10.132.0.16/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-79988fc9d-2hkvw

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

kubelet

router-default-79988fc9d-2hkvw

Created

Created container: router

openshift-ingress

kubelet

router-default-79988fc9d-2hkvw

Started

Started container router

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-5b76b67bf4 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-5b76b67bf4

SuccessfulCreate

Created pod: kube-auth-proxy-5b76b67bf4-xtqs9

openshift-ingress

replicaset-controller

kube-auth-proxy-5b76b67bf4

SuccessfulCreate

Created pod: kube-auth-proxy-5b76b67bf4-tsmb8
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-5b76b67bf4

FailedCreate

Error creating: pods "kube-auth-proxy-5b76b67bf4-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-xtqs9

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-5b76b67bf4-xtqs9

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-868tl

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-tsmb8

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

kube-auth-proxy-5b76b67bf4-tsmb8

AddedInterface

Add eth0 [10.133.0.17/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-868tl

AddedInterface

Add eth0 [10.133.0.18/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-xtqs9

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-xtqs9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.97s (2.97s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-xtqs9

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

Created

Created container: discovery

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-tsmb8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 5.197s (5.197s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-tsmb8

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5b76b67bf4-tsmb8

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.939s (3.939s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

Started

Started container discovery

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-k6ksf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 4.407s (4.407s including waiting). Image size: 181318953 bytes.

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-mv7q9

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-mv7q9

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-mv7q9

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-mv7q9

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-mv7q9

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.682s (2.682s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-mv7q9

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-mv7q9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-mv7q9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-868tl

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-868tl

Killing

Stopping container discovery

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-ht96l

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-ht96l

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.744s (2.744s including waiting). Image size: 181318953 bytes.

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-45sdg

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-cldpb
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-45sdg

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-cldpb

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-cldpb

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-5b54b7fbb7-45sdg

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-cldpb

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-cldpb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.444s (3.444s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-45sdg

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-45sdg

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.136s (3.136s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-45sdg

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-cldpb

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Started

Started container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

Unhealthy

Startup probe failed: Get "http://10.133.0.27:15021/healthz/ready": dial tcp 10.133.0.27:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ht96l

ProbeError

Startup probe error: Get "http://10.133.0.27:15021/healthz/ready": dial tcp 10.133.0.27:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-45sdg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-cldpb

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-74969cc5f8

SuccessfulCreate

Created pod: payload-processing-74969cc5f8-tflk8

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-74969cc5f8 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-558d9b8847 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-558d9b8847

SuccessfulCreate

Created pod: payload-pre-processing-558d9b8847-2lchj

openshift-ingress

multus

payload-pre-processing-558d9b8847-2lchj

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2lchj

Pulling

Pulling image "quay.io/maas/odh-praxis-extproc:pr699-76cb977"

openshift-ingress

multus

payload-processing-74969cc5f8-tflk8

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-74969cc5f8-tflk8

Pulling

Pulling image "quay.io/maas/odh-praxis-extproc:pr699-76cb977"

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2lchj

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-tflk8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2lchj

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-tflk8

Pulled

Successfully pulled image "quay.io/maas/odh-praxis-extproc:pr699-76cb977" in 3.134s (3.134s including waiting). Image size: 120164829 bytes.

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2lchj

Pulled

Successfully pulled image "quay.io/maas/odh-praxis-extproc:pr699-76cb977" in 3.103s (3.103s including waiting). Image size: 120164829 bytes.

openshift-ingress

kubelet

payload-processing-74969cc5f8-tflk8

Started

Started container payload-processing

openshift-ingress

deployment-controller

e2e-worker-w0-74ef1f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-74ef1f-openshift-default-59774bc458 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-7c96ca

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-7c96ca

openshift-ingress

deployment-controller

e2e-worker-w3-7c96ca-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-7c96ca-openshift-default-78cf477b8 from 0 to 1

openshift-ingress

multus

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-74ef1f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-74ef1f

openshift-ingress

gateway_labeler_controller

e2e-inf-a-d5559a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d5559a

openshift-ingress

replicaset-controller

e2e-worker-w0-74ef1f-openshift-default-59774bc458

SuccessfulCreate

Created pod: e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

openshift-ingress

replicaset-controller

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb

SuccessfulCreate

Created pod: e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

openshift-ingress

replicaset-controller

e2e-worker-w3-7c96ca-openshift-default-78cf477b8

SuccessfulCreate

Created pod: e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

openshift-ingress

deployment-controller

e2e-inf-a-d5559a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-d5559a-openshift-default-69f8d5f7fb from 0 to 1

openshift-ingress

multus

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-7c96ca: serviceaccount "payload-processing-e2e-worker-w3-7c96ca" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-7c96ca

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-7c96ca-6c995dd895

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-7c96ca-6c995dd895-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-7c96ca: serviceaccount "payload-processing-e2e-worker-w3-7c96ca" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-7c96ca-6c995dd895

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-7c96ca

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-7c96ca-6c995dd895 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-74ef1f-846448c85c

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-74ef1f: serviceaccount "payload-processing-e2e-worker-w0-74ef1f" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-74ef1f-846448c85c

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-74ef1f-846448c85c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-74ef1f: serviceaccount "payload-processing-e2e-worker-w0-74ef1f" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-74ef1f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-74ef1f-846448c85c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-74ef1f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-d5559a-757b5b5b47

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-d5559a-757b5b5b47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d5559a: serviceaccount "payload-processing-e2e-inf-a-d5559a" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d5559a: serviceaccount "payload-processing-e2e-inf-a-d5559a" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

Created

Created container: payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-d5559a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-d5559a-757b5b5b47 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-d5559a-757b5b5b47

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-d5559a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-d5559a-78db7b4866 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

Started

Started container payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-4df819e5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4df819e5

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

GatewayAuthBootstrap

e2e-ait-4df819e5

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ait-4df819e5-authn-ssl: envoyfilters.networking.istio.io "e2e-ait-4df819e5-authn-ssl" already exists

openshift-ingress

deployment-controller

e2e-ait-4df819e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-4df819e5-openshift-default-6f5df5dbd from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-4df819e5-openshift-default-6f5df5dbd

SuccessfulCreate

Created pod: e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

Created

Created container: payload-pre-processing

openshift-ingress

multus

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-4df819e5-dd8f79879

SuccessfulCreate

Created pod: payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

openshift-ingress

multus

payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-4df819e5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-4df819e5-58dd877898 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-4df819e5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-4df819e5-dd8f79879 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-4df819e5-58dd877898

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-4df819e5-dd8f79879

FailedCreate

Error creating: pods "payload-processing-e2e-ait-4df819e5-dd8f79879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4df819e5: serviceaccount "payload-processing-e2e-ait-4df819e5" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-4df819e5-58dd877898

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-4df819e5-58dd877898-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4df819e5: serviceaccount "payload-processing-e2e-ait-4df819e5" not found

openshift-ingress

kubelet

payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Unhealthy

Readiness probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

ProbeError

Readiness probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Unhealthy

Readiness probe failed: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

ProbeError

Readiness probe error: Get "http://10.132.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Created

Created container: istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-inf-b-08e9c4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-08e9c4

openshift-ingress

multus

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-inf-b-08e9c4-openshift-default-6785bc6c58

SuccessfulCreate

Created pod: e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

openshift-ingress

deployment-controller

e2e-inf-b-08e9c4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-08e9c4-openshift-default-6785bc6c58 from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

openshift-ingress

multus

payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-08e9c4-66776b9969

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-08e9c4-66776b9969-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-08e9c4: serviceaccount "payload-processing-e2e-inf-b-08e9c4" not found
(x2)

openshift-ingress

kubelet

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-08e9c4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-08e9c4-66776b9969

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-08e9c4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-08e9c4-66776b9969 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-08e9c4: serviceaccount "payload-processing-e2e-inf-b-08e9c4" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

Created

Created container: payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

ProbeError

Readiness probe error: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Unhealthy

Readiness probe failed: Get "http://10.133.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-processing-e2e-ait-4df819e5-dd8f79879-kjwbz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4df819e5-58dd877898-544nk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-4df819e5-openshift-default-6f5df5dbd-rx9db

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd

SuccessfulCreate

Created pod: e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

openshift-ingress

deployment-controller

e2e-migrate-813c61ca-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-813c61ca-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-813c61ca-gw
(x2)

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Started

Started container istio-proxy

openshift-ingress

multus

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Unhealthy

Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

ProbeError

Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-813c61ca

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-813c61ca-6669f485db from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-813c61ca: serviceaccount "payload-processing-e2e-migrate-813c61ca" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

openshift-ingress

multus

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-813c61ca-6669f485db

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-813c61ca

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-813c61ca-684cb9f698 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-813c61ca-6669f485db

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-813c61ca-6669f485db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-813c61ca: serviceaccount "payload-processing-e2e-migrate-813c61ca" not found

openshift-ingress

multus

payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-migrate-813c61ca-6669f485db-sv9mk

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-migrate-813c61ca

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-migrate-813c61ca: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-migrate-813c61ca": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-migrate-813c61ca, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f10b4c98-d61e-4ed5-a0d9-4155de99f07f, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-813c61ca-684cb9f698-2wvnt

Killing

Stopping container payload-pre-processing

openshift-ingress

multus

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ait-e886bcb7-openshift-default-7f9995d54b

SuccessfulCreate

Created pod: e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

openshift-ingress

deployment-controller

e2e-ait-e886bcb7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-e886bcb7-openshift-default-7f9995d54b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-e886bcb7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-e886bcb7

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-migrate-813c61ca-gw-openshift-default-85f5bcb5dd-rsk9m

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-processing-e2e-ait-e886bcb7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-e886bcb7-5b6c5d9b45 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45

SuccessfulCreate

Created pod: payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e886bcb7: serviceaccount "payload-processing-e2e-ait-e886bcb7" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-e886bcb7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45

FailedCreate

Error creating: pods "payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e886bcb7: serviceaccount "payload-processing-e2e-ait-e886bcb7" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-e886bcb7-5c94ff68bb-67bj8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-e886bcb7-5b6c5d9b45-k926f

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-b6741af4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-b6741af4-openshift-default-f4c9fb88 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-b6741af4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-b6741af4

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-e886bcb7-openshift-default-7f9995d54b-xcdzz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

replicaset-controller

e2e-derive-b6741af4-openshift-default-f4c9fb88

SuccessfulCreate

Created pod: e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

openshift-ingress

multus

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-b6741af4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-b6741af4-d5bc899bb from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-derive-b6741af4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-b6741af4-f49f6f6b5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-b6741af4-f49f6f6b5

SuccessfulCreate

Created pod: payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-b6741af4: serviceaccount "payload-processing-e2e-derive-b6741af4" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-b6741af4-f49f6f6b5

FailedCreate

Error creating: pods "payload-processing-e2e-derive-b6741af4-f49f6f6b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-b6741af4: serviceaccount "payload-processing-e2e-derive-b6741af4" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

Created

Created container: istio-proxy
(x3)

openshift-ingress

kubelet

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b6741af4-d5bc899bb-kmj6s

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-b6741af4-f49f6f6b5-phwfq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-b6741af4-openshift-default-f4c9fb88-796br

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-mt-84ee0c4c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-84ee0c4c

openshift-ingress

replicaset-controller

e2e-mt-84ee0c4c-openshift-default-65b8bf5584

SuccessfulCreate

Created pod: e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

openshift-ingress

deployment-controller

e2e-mt-84ee0c4c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-84ee0c4c-openshift-default-65b8bf5584 from 0 to 1

openshift-ingress

multus

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-84ee0c4c: serviceaccount "payload-processing-e2e-mt-84ee0c4c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

openshift-ingress

deployment-controller

payload-processing-e2e-mt-84ee0c4c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-84ee0c4c-6fc46956bf from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-84ee0c4c-6fc46956bf

SuccessfulCreate

Created pod: payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-84ee0c4c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-84ee0c4c-6fc46956bf

FailedCreate

Error creating: pods "payload-processing-e2e-mt-84ee0c4c-6fc46956bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-84ee0c4c: serviceaccount "payload-processing-e2e-mt-84ee0c4c" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Unhealthy

Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

ProbeError

Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d5559a-78db7b4866-b9cgp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d5559a-757b5b5b47-clfs7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-84ee0c4c-6b95c5bf9d-6cbvf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-84ee0c4c-6fc46956bf-5zn7r

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-inf-a-d5559a-openshift-default-69f8d5f7fb-w8mq5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-mt-84ee0c4c-openshift-default-65b8bf5584-bzq67

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-08e9c4-7d76f6b6bb-cpbsq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-08e9c4-66776b9969-4kjdj

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w1-4d299f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-4d299f

openshift-ingress

deployment-controller

e2e-shared-a-w1-4d299f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd

SuccessfulCreate

Created pod: e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

openshift-ingress

multus

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-08e9c4-openshift-default-6785bc6c58-jwzxg

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-a-w2-7073b8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-7073b8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-7073b8

openshift-ingress

replicaset-controller

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd

SuccessfulCreate

Created pod: e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-4d299f: serviceaccount "payload-processing-e2e-shared-a-w1-4d299f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

Created

Created container: payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w1-4d299f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w1-4d299f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-4d299f: serviceaccount "payload-processing-e2e-shared-a-w1-4d299f" not found

openshift-ingress

multus

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-7073b8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-7073b8: serviceaccount "payload-processing-e2e-shared-a-w2-7073b8" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-7073b8: serviceaccount "payload-processing-e2e-shared-a-w2-7073b8" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-7073b8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4

SuccessfulCreate

Created pod: e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

openshift-ingress

multus

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-b-w1-501bed-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w1-501bed

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-501bed

openshift-ingress

multus

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648

SuccessfulCreate

Created pod: e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

openshift-ingress

deployment-controller

e2e-shared-b-w2-33ff04-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-33ff04-openshift-default-8698c8b648 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-33ff04

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-33ff04
(x6)

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

ProbeError

Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

Unhealthy

Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-33ff04

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-33ff04: serviceaccount "payload-processing-e2e-shared-b-w2-33ff04" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-33ff04

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-33ff04: serviceaccount "payload-processing-e2e-shared-b-w2-33ff04" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-501bed: serviceaccount "payload-processing-e2e-shared-b-w1-501bed" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w1-501bed

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-501bed-5b66848f59

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w1-501bed

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w1-501bed-5b66848f59 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-501bed-5b66848f59

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w1-501bed-5b66848f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-501bed: serviceaccount "payload-processing-e2e-shared-b-w1-501bed" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

Unhealthy

Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

ProbeError

Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-mt-d2e6cc70

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d2e6cc70

openshift-ingress

deployment-controller

e2e-mt-d2e6cc70-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-d2e6cc70-openshift-default-56775f7484 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-d2e6cc70-openshift-default-56775f7484

SuccessfulCreate

Created pod: e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

openshift-ingress

multus

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-d2e6cc70

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-d2e6cc70-85747db998 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d2e6cc70-85747db998

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d2e6cc70-7cf55bd589

SuccessfulCreate

Created pod: payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d2e6cc70-7cf55bd589

FailedCreate

Error creating: pods "payload-processing-e2e-mt-d2e6cc70-7cf55bd589-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d2e6cc70: serviceaccount "payload-processing-e2e-mt-d2e6cc70" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-mt-d2e6cc70

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-d2e6cc70-7cf55bd589 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d2e6cc70-85747db998

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-d2e6cc70-85747db998-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d2e6cc70: serviceaccount "payload-processing-e2e-mt-d2e6cc70" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-7c96ca-76bbd5f94-qnbtl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-7c96ca-6c995dd895-tv8sl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w3-7c96ca-openshift-default-78cf477b8-mkhz5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

Unhealthy

Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

ProbeError

Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2e6cc70-85747db998-crqwq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d2e6cc70-7cf55bd589-6rxc9

Unhealthy

Readiness probe failed: dial tcp 10.133.0.61:9004: i/o timeout

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-74ef1f-695df95b6-vp2xp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-74ef1f-846448c85c-g7nnp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-74ef1f-openshift-default-59774bc458-pfk99

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-d2e6cc70-openshift-default-56775f7484-sgp88

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-api-a-910e11-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-910e11-openshift-default-6b47698bc4 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-910e11-openshift-default-6b47698bc4

SuccessfulCreate

Created pod: e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

openshift-ingress

multus

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-api-a-910e11

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-910e11

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-910e11

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-910e11-6cc584fc58 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-910e11-6cc584fc58

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-910e11-6cc584fc58

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-910e11-6cc584fc58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-910e11: serviceaccount "payload-processing-e2e-api-a-910e11" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-910e11-996975df7

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-910e11-996975df7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-910e11: serviceaccount "payload-processing-e2e-api-a-910e11" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-910e11-996975df7

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-910e11-996975df7-287b8

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-910e11

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-910e11-996975df7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-910e11-996975df7-287b8

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-a-910e11-996975df7-287b8

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-910e11-996975df7-287b8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-910e11-996975df7-287b8

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

ProbeError

Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Unhealthy

Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-api-b-0a7ab7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-0a7ab7

openshift-ingress

deployment-controller

e2e-api-b-0a7ab7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-0a7ab7-openshift-default-67697547c4 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-b-0a7ab7-openshift-default-67697547c4

SuccessfulCreate

Created pod: e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

openshift-ingress

multus

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

AddedInterface

Add eth0 [10.132.0.90/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-0a7ab7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-0a7ab7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-0a7ab7-7bc9b64f77 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0a7ab7: serviceaccount "payload-processing-e2e-api-b-0a7ab7" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0a7ab7: serviceaccount "payload-processing-e2e-api-b-0a7ab7" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

Unhealthy

Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

ProbeError

Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-910e11-6cc584fc58-tjwh7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-910e11-996975df7-287b8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-a-910e11-openshift-default-6b47698bc4-4xb8t

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0a7ab7-799db6d58b-4jrpr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-0a7ab7-7bc9b64f77-gkmt5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-0a7ab7-openshift-default-67697547c4-whgjq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-4d299f-bfb7458f9-fdfhb

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-a-w1-4d299f

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-w1-4d299f: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-w1-4d299f": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-w1-4d299f, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 63e9efde-5f19-425d-a9bc-c06b6a0b9cad, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-4d299f-6bffb97986-9gcpp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-a-w1-4d299f-openshift-default-5677ddf4bd-2x25l

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-501bed-5b66848f59-x76qn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-501bed-549f686bcd-q5sgm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w1-501bed-openshift-default-6bf8d847d4-4rlgr

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ipp-a-6eb41d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-6eb41d-openshift-default-8b94d9b66 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-6eb41d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-6eb41d

openshift-ingress

replicaset-controller

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66

SuccessfulCreate

Created pod: e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

openshift-ingress

multus

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

AddedInterface

Add eth0 [10.132.0.95/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-6eb41d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-6eb41d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-6eb41d: serviceaccount "payload-processing-e2e-ipp-a-6eb41d" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-6eb41d: serviceaccount "payload-processing-e2e-ipp-a-6eb41d" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

AddedInterface

Add eth0 [10.132.0.97/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Unhealthy

Readiness probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

ProbeError

Readiness probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Unhealthy

Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

ProbeError

Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-25b1b3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-25b1b3

openshift-ingress

multus

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

AddedInterface

Add eth0 [10.132.0.98/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ipp-b-25b1b3-openshift-default-54975cf586

SuccessfulCreate

Created pod: e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

openshift-ingress

deployment-controller

e2e-ipp-b-25b1b3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-25b1b3-openshift-default-54975cf586 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-25b1b3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

AddedInterface

Add eth0 [10.133.0.73/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-25b1b3: serviceaccount "payload-processing-e2e-ipp-b-25b1b3" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

openshift-ingress

multus

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

AddedInterface

Add eth0 [10.132.0.99/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-25b1b3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-25b1b3: serviceaccount "payload-processing-e2e-ipp-b-25b1b3" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

ProbeError

Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

Unhealthy

Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-ipp-cleanup-93b6ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd from 0 to 1

openshift-ingress

multus

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-93b6ef

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-93b6ef

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd

SuccessfulCreate

Created pod: e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-93b6ef: serviceaccount "payload-processing-e2e-ipp-cleanup-93b6ef" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-93b6ef

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-93b6ef

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-93b6ef: serviceaccount "payload-processing-e2e-ipp-cleanup-93b6ef" not found

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

AddedInterface

Add eth0 [10.133.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

AddedInterface

Add eth0 [10.132.0.100/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-93b6ef-798fcc449-ftnss

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-93b6ef-dfc477694-strfm

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-cleanup-93b6ef

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-93b6ef: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-cleanup-93b6ef": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-93b6ef, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 29a94d89-36b1-40c5-a07e-c8a9c06416f5, UID in object meta:

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-cleanup-93b6ef-openshift-default-7d4d79f9cd-n8b2n

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-6eb41d-77f5cf5fc7-jdc8h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-6eb41d-6d8f88b6bb-jlrfg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-6eb41d-openshift-default-8b94d9b66-xn552

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-25b1b3-5967ccd554-tjbbh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-25b1b3-5fbb75f6c7-r6dq7

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-b-25b1b3

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-25b1b3: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-25b1b3": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-25b1b3, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a102dd5e-2bd0-45dd-8dd6-ffe531c8e11e, UID in object meta:

openshift-ingress

kubelet

e2e-ipp-b-25b1b3-openshift-default-54975cf586-v6p5d

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-7073b8-6fbb8b8d7-x2bgp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-7073b8-6b6c986694-dzcp6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w2-7073b8-openshift-default-74b5b585bd-bpknx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-33ff04-7fb47dd7dc-scd64

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-33ff04-87579c798-xvxc9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-33ff04-openshift-default-8698c8b648-hhk8l

Killing

Stopping container istio-proxy
(x20)

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2lchj

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (payload-processing-dockercfg-zkwlt); attempting to pull the image may not succeed.
(x20)

openshift-ingress

kubelet

payload-processing-74969cc5f8-tflk8

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (payload-processing-dockercfg-zkwlt); attempting to pull the image may not succeed.