Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf to ip-10-0-129-142.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-s5vnk

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-s5vnk to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht to ip-10-0-142-48.ec2.internal

openshift-ingress

kube-auth-proxy-79f4c98848-bqq9k

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-79f4c98848-bqq9k to ip-10-0-142-48.ec2.internal

openshift-ingress

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

Scheduled

Successfully assigned openshift-ingress/e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-kxd7h

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-kxd7h to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Scheduled

Successfully assigned openshift-ingress/e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29 to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9 to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952 to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27 to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh to ip-10-0-142-48.ec2.internal

openshift-ingress

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Scheduled

Successfully assigned openshift-ingress/e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79 to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx to ip-10-0-142-48.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-476rf

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-476rf to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6 to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59 to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-autores-8b875d-949df588f-m2cmk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-8b875d-949df588f-m2cmk to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz to ip-10-0-142-48.ec2.internal

openshift-ingress

kube-auth-proxy-79f4c98848-jnvlv

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-79f4c98848-jnvlv to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb to ip-10-0-142-63.ec2.internal

openshift-ingress

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5 to ip-10-0-142-63.ec2.internal

openshift-ingress

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

Scheduled

Successfully assigned openshift-ingress/e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b to ip-10-0-129-142.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-2mcsv to ip-10-0-142-48.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799crdmz4 to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f to ip-10-0-142-63.ec2.internal

openshift-ingress

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Scheduled

Successfully assigned openshift-ingress/e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst to ip-10-0-142-48.ec2.internal

openshift-ingress

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np to ip-10-0-142-48.ec2.internal

openshift-ingress

router-default-c65ff548d-bxgzt

Scheduled

Successfully assigned openshift-ingress/router-default-c65ff548d-bxgzt to ip-10-0-142-48.ec2.internal

openshift-ingress

router-default-c65ff548d-bxgzt

FailedScheduling

0/3 nodes are available: 3 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

openshift-ingress

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-vr6p8

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-vr6p8 to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww to ip-10-0-129-142.ec2.internal

openshift-ingress

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5 to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8 to ip-10-0-142-63.ec2.internal

openshift-ingress

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

Scheduled

Successfully assigned openshift-ingress/e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp to ip-10-0-142-48.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4 to ip-10-0-142-63.ec2.internal

openshift-ingress

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Scheduled

Successfully assigned openshift-ingress/e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx to ip-10-0-142-63.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c to ip-10-0-129-142.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw to ip-10-0-142-63.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-c65ff548d from 0 to 1
(x11)

openshift-ingress

replicaset-controller

router-default-c65ff548d

FailedCreate

Error creating: pods "router-default-c65ff548d-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-c65ff548d

FailedCreate

Error creating: pods "router-default-c65ff548d-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-c65ff548d

SuccessfulCreate

Created pod: router-default-c65ff548d-bxgzt
(x6)

openshift-ingress

kubelet

router-default-c65ff548d-bxgzt

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-c65ff548d-bxgzt

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-c65ff548d-bxgzt

Started

Started container router

openshift-ingress

multus

router-default-c65ff548d-bxgzt

AddedInterface

Add eth0 [10.132.0.12/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-c65ff548d-bxgzt

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

kubelet

router-default-c65ff548d-bxgzt

Created

Created container: router
(x2)

openshift-ingress

replicaset-controller

kube-auth-proxy-79f4c98848

FailedCreate

Error creating: pods "kube-auth-proxy-79f4c98848-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-79f4c98848 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-79f4c98848

SuccessfulCreate

Created pod: kube-auth-proxy-79f4c98848-bqq9k

openshift-ingress

replicaset-controller

kube-auth-proxy-79f4c98848

SuccessfulCreate

Created pod: kube-auth-proxy-79f4c98848-jnvlv

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

kube-auth-proxy-79f4c98848-jnvlv

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-ingress

multus

kube-auth-proxy-79f4c98848-bqq9k

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-bqq9k

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-s5vnk

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-jnvlv

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-s5vnk

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-bqq9k

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.683s (2.683s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-jnvlv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.686s (2.686s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-jnvlv

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-jnvlv

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-bqq9k

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-79f4c98848-bqq9k

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.947s (2.947s including waiting). Image size: 181126104 bytes.

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-68cf4d799c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-68cf4d799c from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799crdmz4

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 3.277s (3.277s including waiting). Image size: 181318953 bytes.

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-476rf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-476rf

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-476rf

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-476rf

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-476rf

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-476rf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.258s (2.258s including waiting). Image size: 181126104 bytes.

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-s5vnk

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-s5vnk

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5ccb968c66

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5ccb968c66-2mcsv

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5ccb968c66 from 0 to 1

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

multus

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-vr6p8

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-kxd7h
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-vr6p8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-5b54b7fbb7-vr6p8

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kxd7h

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-kxd7h

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-vr6p8

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-vr6p8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.116s (2.116s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-vr6p8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kxd7h

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.582s (3.582s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kxd7h

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kxd7h

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

ProbeError

Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-2mcsv

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-inf-a-4ab744

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-4ab744

openshift-ingress

deployment-controller

e2e-inf-a-4ab744-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-4ab744-openshift-default-66fdc6978b from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w0-909a40-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-909a40-openshift-default-5d6dcfd97d from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d

SuccessfulCreate

Created pod: e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

openshift-ingress

replicaset-controller

e2e-inf-a-4ab744-openshift-default-66fdc6978b

SuccessfulCreate

Created pod: e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-d8a600

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-d8a600

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-909a40

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-909a40

openshift-ingress

multus

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w4-d8a600-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-d8a600-openshift-default-9d45d7c78 from 0 to 1

openshift-ingress

multus

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-d8a600-openshift-default-9d45d7c78

SuccessfulCreate

Created pod: e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

openshift-ingress

multus

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-4ab744

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-4ab744-66bf944c97 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-4ab744

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-4ab744-5868949898 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-4ab744-5868949898

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-4ab744-5868949898-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4ab744: serviceaccount "payload-processing-e2e-inf-a-4ab744" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-4ab744-5868949898

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-4ab744-66bf944c97

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-4ab744-66bf944c97

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-4ab744-66bf944c97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-4ab744: serviceaccount "payload-processing-e2e-inf-a-4ab744" not found

openshift-ingress

multus

payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-909a40

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-909a40-5449d8667c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-909a40

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-909a40-5449d8667c

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-909a40-5449d8667c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-909a40: serviceaccount "payload-processing-e2e-worker-w0-909a40" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-909a40-5449d8667c

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-909a40: serviceaccount "payload-processing-e2e-worker-w0-909a40" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl

Started

Started container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-autores-8b875d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-8b875d

openshift-ingress

deployment-controller

e2e-autores-8b875d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-8b875d-openshift-default-75c4bf76d6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-autores-8b875d-openshift-default-75c4bf76d6

SuccessfulCreate

Created pod: e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-d8a600

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d from 0 to 1

openshift-ingress

multus

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-d8a600

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-d8a600-557f86fc65 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-d8a600-557f86fc65

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-d8a600: serviceaccount "payload-processing-e2e-worker-w4-d8a600" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-d8a600-557f86fc65

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-d8a600-557f86fc65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-d8a600: serviceaccount "payload-processing-e2e-worker-w4-d8a600" not found

openshift-ingress

multus

payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

Created

Created container: payload-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-119aafd2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-119aafd2

openshift-ingress

GatewayAuthBootstrap

e2e-ait-119aafd2

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ait-119aafd2-authn-ssl: envoyfilters.networking.istio.io "e2e-ait-119aafd2-authn-ssl" already exists

openshift-ingress

deployment-controller

e2e-ait-119aafd2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-119aafd2-openshift-default-8599c89cc4 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-119aafd2-openshift-default-8599c89cc4

SuccessfulCreate

Created pod: e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79
(x2)

openshift-ingress

kubelet

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

Created

Created container: istio-proxy

openshift-ingress

multus

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

ProbeError

Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Unhealthy

Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

ProbeError

Readiness probe error: Get "http://10.132.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Unhealthy

Readiness probe failed: Get "http://10.132.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

gateway_labeler_controller

e2e-inf-b-6ef84e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-6ef84e
(x2)

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

e2e-inf-b-6ef84e-openshift-default-857958fcbd

SuccessfulCreate

Created pod: e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

openshift-ingress

deployment-controller

e2e-inf-b-6ef84e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-6ef84e-openshift-default-857958fcbd from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Started

Started container istio-proxy

openshift-ingress

multus

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-8b875d-998947cf4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-8b875d-998947cf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-8b875d: serviceaccount "payload-processing-e2e-autores-8b875d" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-8b875d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-8b875d-998947cf4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-autores-8b875d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-8b875d-949df588f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-8b875d-949df588f

SuccessfulCreate

Created pod: payload-processing-e2e-autores-8b875d-949df588f-m2cmk
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-8b875d-949df588f

FailedCreate

Error creating: pods "payload-processing-e2e-autores-8b875d-949df588f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-8b875d: serviceaccount "payload-processing-e2e-autores-8b875d" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-autores-8b875d-949df588f-m2cmk

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-8b875d-998947cf4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-8b875d-949df588f-m2cmk

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-8b875d-949df588f-m2cmk

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-8b875d-949df588f-m2cmk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

ProbeError

Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Unhealthy

Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

ProbeError

Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Unhealthy

Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-119aafd2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-119aafd2: serviceaccount "payload-processing-e2e-ait-119aafd2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-119aafd2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-119aafd2-749c6d5549 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-119aafd2-749c6d5549

FailedCreate

Error creating: pods "payload-processing-e2e-ait-119aafd2-749c6d5549-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-119aafd2: serviceaccount "payload-processing-e2e-ait-119aafd2" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

openshift-ingress

multus

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-119aafd2-749c6d5549

SuccessfulCreate

Created pod: payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-6ef84e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-6ef84e-5776cb84c5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-6ef84e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6ef84e: serviceaccount "payload-processing-e2e-inf-b-6ef84e" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-6ef84e-5776cb84c5

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-6ef84e-5776cb84c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6ef84e: serviceaccount "payload-processing-e2e-inf-b-6ef84e" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-6ef84e-5776cb84c5

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

ProbeError

Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

Unhealthy

Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ait-119aafd2-749c6d5549-kxnwt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-119aafd2-68f7df6d7d-xvd5v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ait-119aafd2-openshift-default-8599c89cc4-47k79

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

gateway_labeler_controller

e2e-migrate-ad2bfa56-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-ad2bfa56-gw

openshift-ingress

deployment-controller

e2e-migrate-ad2bfa56-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88 from 0 to 1

openshift-ingress

replicaset-controller

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88

SuccessfulCreate

Created pod: e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

openshift-ingress

multus

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-ad2bfa56: serviceaccount "payload-processing-e2e-migrate-ad2bfa56" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-ad2bfa56: serviceaccount "payload-processing-e2e-migrate-ad2bfa56" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-ad2bfa56

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-ad2bfa56

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-ad2bfa56-9d9fc497d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

openshift-ingress

kubelet

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b
(x2)

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Created

Created container: istio-proxy
(x6)

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-migrate-ad2bfa56-9d9fc497d-x7b8b

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-ad2bfa56-6d7784cc94-s7x59

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-ait-019b7d43-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-019b7d43-openshift-default-7d979df69b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-019b7d43

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-019b7d43

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-019b7d43-openshift-default-7d979df69b

SuccessfulCreate

Created pod: e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

openshift-ingress

multus

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-migrate-ad2bfa56-gw-openshift-default-85f877fd88-lzhht

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 12.372s (12.372s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

payload-processing-e2e-ait-019b7d43

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-019b7d43-69869f47c5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-019b7d43-69869f47c5

FailedCreate

Error creating: pods "payload-processing-e2e-ait-019b7d43-69869f47c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-019b7d43: serviceaccount "payload-processing-e2e-ait-019b7d43" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-019b7d43

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-019b7d43-74bd595bdb from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-019b7d43-69869f47c5

SuccessfulCreate

Created pod: payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-019b7d43: serviceaccount "payload-processing-e2e-ait-019b7d43" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-8b875d-949df588f-m2cmk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-8b875d-998947cf4-sbvhf

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Unhealthy

Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

ProbeError

Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-autores-8b875d-openshift-default-75c4bf76d6-2ddkp

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-explicit-efa70c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-efa70c

openshift-ingress

deployment-controller

e2e-explicit-efa70c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-efa70c-openshift-default-689cf88f48 from 0 to 1

openshift-ingress

replicaset-controller

e2e-explicit-efa70c-openshift-default-689cf88f48

SuccessfulCreate

Created pod: e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

openshift-ingress

multus

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-019b7d43-74bd595bdb-94fzd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-019b7d43-69869f47c5-8sj9k

Killing

Stopping container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-efa70c-9fff7bf45

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-efa70c-9fff7bf45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-efa70c: serviceaccount "payload-processing-e2e-explicit-efa70c" not found

openshift-ingress

kubelet

payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-efa70c: serviceaccount "payload-processing-e2e-explicit-efa70c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-efa70c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-efa70c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-efa70c-9fff7bf45 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-019b7d43-openshift-default-7d979df69b-4hzrn

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-efa70c-9fff7bf45

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

openshift-ingress

multus

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

e2e-derive-b95e3be6-openshift-default-7c8f778f88

SuccessfulCreate

Created pod: e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

openshift-ingress

kubelet

payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

Created

Created container: payload-pre-processing

openshift-ingress

deployment-controller

e2e-derive-b95e3be6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-b95e3be6-openshift-default-7c8f778f88 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-b95e3be6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-b95e3be6

openshift-ingress

multus

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-b95e3be6-5c99df9665

FailedCreate

Error creating: pods "payload-processing-e2e-derive-b95e3be6-5c99df9665-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-b95e3be6: serviceaccount "payload-processing-e2e-derive-b95e3be6" not found

openshift-ingress

deployment-controller

payload-processing-e2e-derive-b95e3be6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-b95e3be6-5c99df9665 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-b95e3be6-5c99df9665

SuccessfulCreate

Created pod: payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-b95e3be6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-b95e3be6: serviceaccount "payload-processing-e2e-derive-b95e3be6" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

openshift-ingress

kubelet

payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x7)

openshift-ingress

kubelet

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

ProbeError

Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

Unhealthy

Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Unhealthy

Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

ProbeError

Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-derive-b95e3be6-5c99df9665-6f7rj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b95e3be6-5997c7ddc8-n5bs4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-derive-b95e3be6-openshift-default-dockercfg-pghsz); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-derive-b95e3be6-openshift-default-7c8f778f88-krbst

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-efa70c-5dffc9cf54-kh2fw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-efa70c-9fff7bf45-spx2h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-explicit-efa70c-openshift-default-689cf88f48-gcbv6

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-noten-c8b23c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-c8b23c-openshift-default-c48fd5fd6 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-noten-c8b23c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-c8b23c

openshift-ingress

GatewayAuthBootstrap

e2e-noten-c8b23c

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-noten-c8b23c-authn-ssl: envoyfilters.networking.istio.io "e2e-noten-c8b23c-authn-ssl" already exists

openshift-ingress

kubelet

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

multus

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-noten-c8b23c-openshift-default-c48fd5fd6

SuccessfulCreate

Created pod: e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

openshift-ingress

kubelet

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

Created

Created container: istio-proxy

openshift-ingress

deployment-controller

e2e-mt-58beb43a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-58beb43a-openshift-default-7745cc49d8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-58beb43a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-58beb43a

openshift-ingress

replicaset-controller

e2e-mt-58beb43a-openshift-default-7745cc49d8

SuccessfulCreate

Created pod: e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

openshift-ingress

multus

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-58beb43a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-58beb43a-57f8c48858 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-58beb43a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-58beb43a-6d467f6769 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-58beb43a-57f8c48858

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-58beb43a-57f8c48858-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-58beb43a: serviceaccount "payload-processing-e2e-mt-58beb43a" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-58beb43a-57f8c48858

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

Created

Created container: payload-pre-processing
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-58beb43a-6d467f6769

FailedCreate

Error creating: pods "payload-processing-e2e-mt-58beb43a-6d467f6769-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-58beb43a: serviceaccount "payload-processing-e2e-mt-58beb43a" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-58beb43a-6d467f6769

SuccessfulCreate

Created pod: payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

openshift-ingress

kubelet

payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Unhealthy

Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

ProbeError

Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-58beb43a-57f8c48858-smkxh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-58beb43a-6d467f6769-zhkgd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-58beb43a-openshift-default-7745cc49d8-g66vx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-noten-c8b23c-openshift-default-c48fd5fd6-rtw2q

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-04899e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-04899e

openshift-ingress

replicaset-controller

e2e-shared-a-w2-04899e-openshift-default-5c7d579844

SuccessfulCreate

Created pod: e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

openshift-ingress

multus

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-a-w2-04899e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-04899e-openshift-default-5c7d579844 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-4ab744-66bf944c97-dqqww

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-4ab744-5868949898-n9l2c

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-04899e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-04899e-577ff94d7f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-04899e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

Created

Created container: payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-04899e: serviceaccount "payload-processing-e2e-shared-a-w2-04899e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

Started

Started container payload-pre-processing
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-04899e: serviceaccount "payload-processing-e2e-shared-a-w2-04899e" not found

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-inf-a-4ab744-openshift-default-66fdc6978b-t49np

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-520999

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-520999

openshift-ingress

deployment-controller

e2e-shared-b-w2-520999-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-520999-openshift-default-5559fcc6d8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8

SuccessfulCreate

Created pod: e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

openshift-ingress

multus

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-6ef84e-6c8c79b778-9n9nm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-6ef84e-5776cb84c5-8tmcl

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-520999

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-520999-759594459 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-520999: serviceaccount "payload-processing-e2e-shared-b-w2-520999" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-520999-759594459

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-520999-759594459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-520999: serviceaccount "payload-processing-e2e-shared-b-w2-520999" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-520999

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-520999-759594459

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : failed to sync configmap cache: timed out waiting for the condition

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

Created

Created container: payload-pre-processing
(x7)

openshift-ingress

kubelet

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused

openshift-ingress

kubelet

e2e-inf-b-6ef84e-openshift-default-857958fcbd-gwmnz

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-shared-a-w3-8edc30-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-8edc30

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-8edc30

openshift-ingress

replicaset-controller

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b

SuccessfulCreate

Created pod: e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

openshift-ingress

multus

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-8edc30

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30: serviceaccount "payload-processing-e2e-shared-a-w3-8edc30" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-8edc30

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-8edc30-648d7df86f from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-8edc30: serviceaccount "payload-processing-e2e-shared-a-w3-8edc30" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

Started

Started container payload-processing

openshift-ingress

replicaset-controller

e2e-mt-ab83f76b-openshift-default-c8d959f6

SuccessfulCreate

Created pod: e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

openshift-ingress

deployment-controller

e2e-mt-ab83f76b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-ab83f76b-openshift-default-c8d959f6 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-ab83f76b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-ab83f76b

openshift-ingress

multus

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f

FailedCreate

Error creating: pods "payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ab83f76b: serviceaccount "payload-processing-e2e-mt-ab83f76b" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-ab83f76b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ab83f76b: serviceaccount "payload-processing-e2e-mt-ab83f76b" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-ab83f76b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f

SuccessfulCreate

Created pod: payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

openshift-ingress

kubelet

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb

SuccessfulCreate

Created pod: e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-ee3af0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-ee3af0

openshift-ingress

deployment-controller

e2e-shared-b-w3-ee3af0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb from 0 to 1

openshift-ingress

multus

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

AddedInterface

Add eth0 [10.132.0.83/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-ee3af0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-ee3af0: serviceaccount "payload-processing-e2e-shared-b-w3-ee3af0" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-ee3af0: serviceaccount "payload-processing-e2e-shared-b-w3-ee3af0" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-ee3af0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

Unhealthy

Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

ProbeError

Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

ProbeError

Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

Unhealthy

Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Unhealthy

Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

ProbeError

Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-909a40-cb5f459cd-gjtkn

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-909a40-5449d8667c-hcgsl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-909a40-openshift-default-5d6dcfd97d-7ss27

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-mt-ab83f76b-7f5cbf9b4f-whfx8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ab83f76b-5fc8df9666-lltkg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-8edc30-7988f797db-5gq5z

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-8edc30-648d7df86f-2bjbl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w3-8edc30-openshift-default-69c569bf6b-jn9gb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-ab83f76b-openshift-default-c8d959f6-zjrrb

Killing

Stopping container istio-proxy

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w3-ee3af0

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w3-ee3af0": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w3-ee3af0, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 71e9b3a3-4f14-4796-a988-ef7c9c679a9b, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-ee3af0-7dcd948b5d-7xgbr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-ee3af0-5856f4d85-krddq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w3-ee3af0-openshift-default-6997d6dfbb-gx952

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

e2e-api-a-1f16eb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-1f16eb-openshift-default-85b889d786 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-a-1f16eb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-1f16eb

openshift-ingress

replicaset-controller

e2e-api-a-1f16eb-openshift-default-85b889d786

SuccessfulCreate

Created pod: e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

openshift-ingress

GatewayAuthBootstrap

e2e-api-a-1f16eb

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-api-a-1f16eb-authn-ssl: envoyfilters.networking.istio.io "e2e-api-a-1f16eb-authn-ssl" already exists

openshift-ingress

multus

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-1f16eb-c98dc7757

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-1f16eb-c98dc7757-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1f16eb: serviceaccount "payload-processing-e2e-api-a-1f16eb" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-1f16eb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-1f16eb-c98dc7757 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1f16eb: serviceaccount "payload-processing-e2e-api-a-1f16eb" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-1f16eb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-1f16eb-c98dc7757

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm
(x2)

openshift-ingress

kubelet

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

Unhealthy

Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

ProbeError

Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-api-b-4dfacb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-4dfacb-openshift-default-68989dccf4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-4dfacb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-4dfacb

openshift-ingress

replicaset-controller

e2e-api-b-4dfacb-openshift-default-68989dccf4

SuccessfulCreate

Created pod: e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

openshift-ingress

multus

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

AddedInterface

Add eth0 [10.132.0.98/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-4dfacb-747f67785d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-4dfacb-747f67785d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-4dfacb: serviceaccount "payload-processing-e2e-api-b-4dfacb" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-4dfacb-747f67785d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-4dfacb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-4dfacb-747f67785d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

Created

Created container: payload-pre-processing
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-4dfacb-7fd94db97

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-4dfacb-7fd94db97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-4dfacb: serviceaccount "payload-processing-e2e-api-b-4dfacb" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-4dfacb-7fd94db97

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-4dfacb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-4dfacb-7fd94db97 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-d8a600-8479f5885d-5bvln

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-worker-w4-d8a600

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-worker-w4-d8a600": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-worker-w4-d8a600, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: eef8b2b7-5f86-4e13-846e-7b3835a421e5, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-d8a600-557f86fc65-hwmbf

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Unhealthy

Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

ProbeError

Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-d8a600-openshift-default-9d45d7c78-d7hrh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-api-a-1f16eb-c98dc7757-fxqjp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-1f16eb-c95bf7c74-xpksm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-a-1f16eb-openshift-default-85b889d786-pzb8x

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-4dfacb-747f67785d-22j9p

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-4dfacb-7fd94db97-tn84n

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

ProbeError

Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-api-b-4dfacb-openshift-default-68989dccf4-lqb7f

Unhealthy

Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-04899e-577ff94d7f-8gwn5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-04899e-5f564bd746-lvvx9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w2-04899e-openshift-default-5c7d579844-tpcz5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-520999-cd5b57798-dgb29

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-520999-759594459-bbj6f

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w2-520999-openshift-default-5559fcc6d8-zbzlx

Killing

Stopping container istio-proxy