Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d to ip-10-0-129-114.ec2.internal

openshift-ingress

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44 to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9 to ip-10-0-129-114.ec2.internal

openshift-ingress

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Scheduled

Successfully assigned openshift-ingress/e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq to ip-10-0-142-20.ec2.internal

openshift-ingress

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz to ip-10-0-129-114.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-ltgk2 to ip-10-0-142-20.ec2.internal

openshift-ingress

e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

Scheduled

Successfully assigned openshift-ingress/e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx to ip-10-0-142-20.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-flfb7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-flfb7 to ip-10-0-129-114.ec2.internal

openshift-ingress

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv to ip-10-0-129-114.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k to ip-10-0-142-20.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Scheduled

Successfully assigned openshift-ingress/e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967 to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-b3108b-9449f7f77-z27gg to ip-10-0-138-71.ec2.internal

openshift-ingress

kube-auth-proxy-5dd7bdc797-q87x9

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5dd7bdc797-q87x9 to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr to ip-10-0-129-114.ec2.internal

openshift-ingress

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Scheduled

Successfully assigned openshift-ingress/e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8 to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89 to ip-10-0-142-20.ec2.internal

openshift-ingress

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4 to ip-10-0-129-114.ec2.internal

openshift-ingress

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66 to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl to ip-10-0-138-71.ec2.internal

openshift-ingress

kube-auth-proxy-5dd7bdc797-jljr6

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5dd7bdc797-jljr6 to ip-10-0-142-20.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Scheduled

Successfully assigned openshift-ingress/e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc to ip-10-0-142-20.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcnsz79 to ip-10-0-142-20.ec2.internal

openshift-ingress

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v to ip-10-0-129-114.ec2.internal

openshift-ingress

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx to ip-10-0-142-20.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-p596x

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-p596x to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-wtc2c

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-wtc2c to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Scheduled

Successfully assigned openshift-ingress/e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d to ip-10-0-142-20.ec2.internal

openshift-ingress

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw to ip-10-0-138-71.ec2.internal

openshift-ingress

router-default-7c595d49d-6t78r

Scheduled

Successfully assigned openshift-ingress/router-default-7c595d49d-6t78r to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9 to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6 to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-mt-c615f98e-768d58678f-klqfn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-c615f98e-768d58678f-klqfn to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-80c77821-774f544854-wf28t

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-80c77821-774f544854-wf28t to ip-10-0-138-71.ec2.internal

openshift-ingress

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7 to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4 to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw to ip-10-0-138-71.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h to ip-10-0-138-71.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-gxwvd

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-gxwvd to ip-10-0-142-20.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk to ip-10-0-129-114.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf to ip-10-0-129-114.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-7c595d49d from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer
(x14)

openshift-ingress

replicaset-controller

router-default-7c595d49d

FailedCreate

Error creating: pods "router-default-7c595d49d-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-7c595d49d

FailedCreate

Error creating: pods "router-default-7c595d49d-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-7c595d49d

SuccessfulCreate

Created pod: router-default-7c595d49d-6t78r

openshift-ingress

taint-eviction-controller

router-default-7c595d49d-6t78r

TaintManagerEviction

Cancelling deletion of Pod openshift-ingress/router-default-7c595d49d-6t78r
(x6)

openshift-ingress

kubelet

router-default-7c595d49d-6t78r

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x6)

openshift-ingress

kubelet

router-default-7c595d49d-6t78r

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-7c595d49d-6t78r

Created

Created container: router

openshift-ingress

kubelet

router-default-7c595d49d-6t78r

Started

Started container router

openshift-ingress

kubelet

router-default-7c595d49d-6t78r

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:da130d90f0bffd5c5876c94544c91b2a1be0d23ce2747ddfeafe88e03ee7d8d1" already present on machine

openshift-ingress

multus

router-default-7c595d49d-6t78r

AddedInterface

Add eth0 [10.134.0.10/23] from ovn-kubernetes

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-5dd7bdc797 from 0 to 2

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-jljr6

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-q87x9

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-5dd7bdc797

SuccessfulCreate

Created pod: kube-auth-proxy-5dd7bdc797-q87x9

openshift-ingress

replicaset-controller

kube-auth-proxy-5dd7bdc797

SuccessfulCreate

Created pod: kube-auth-proxy-5dd7bdc797-jljr6
(x5)

openshift-ingress

replicaset-controller

kube-auth-proxy-5dd7bdc797

FailedCreate

Error creating: pods "kube-auth-proxy-5dd7bdc797-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-q87x9

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-gxwvd

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-gxwvd

openshift-ingress

multus

kube-auth-proxy-5dd7bdc797-jljr6

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-jljr6

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

kube-auth-proxy-5dd7bdc797-q87x9

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-q87x9

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-q87x9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 3.594s (3.594s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-q87x9

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

Started

Started container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 5.788s (5.788s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-jljr6

Started

Started container kube-auth-proxy

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-p596x

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-jljr6

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5dd7bdc797-jljr6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 6.45s (6.45s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-p596x

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-p596x

AddedInterface

Add eth0 [10.134.0.19/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-6745bb66bc from 0 to 1

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-6745bb66bc

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-p596x

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-p596x

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-p596x

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.329s (2.329s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-gxwvd

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-gxwvd

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.76s (2.76s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-v2h4k

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.86s (2.86s including waiting). Image size: 181318953 bytes.

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-6745bb66bc

SuccessfulDelete

Deleted pod: data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled down replica set data-science-gateway-data-science-gateway-class-6745bb66bc from 1 to 0

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

ProbeError

Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-6745bb66bcnsz79

Unhealthy

Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-ltgk2

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-wtc2c

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-flfb7

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-flfb7

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-5b54b7fbb7-wtc2c

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wtc2c

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-flfb7

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wtc2c

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wtc2c

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.701s (1.701s including waiting). Image size: 166449337 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-wtc2c

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-flfb7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.489s (2.489s including waiting). Image size: 166449337 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-flfb7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-flfb7

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

ProbeError

Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-ltgk2

Unhealthy

Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-inf-a-8300b9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-8300b9

openshift-ingress

deployment-controller

e2e-inf-a-8300b9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-8300b9-openshift-default-6fd8579f5d from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-a-8300b9-openshift-default-6fd8579f5d

SuccessfulCreate

Created pod: e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

openshift-ingress

multus

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-8300b9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-8300b9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-8300b9-855c8cfdc7 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-8300b9: serviceaccount "payload-processing-e2e-inf-a-8300b9" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

openshift-ingress

multus

payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-8300b9-855c8cfdc7

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-8300b9-855c8cfdc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-8300b9: serviceaccount "payload-processing-e2e-inf-a-8300b9" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-8300b9-855c8cfdc7

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

openshift-ingress

deployment-controller

e2e-autores-b3108b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-b3108b-openshift-default-5d75b95f4b from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

Created

Created container: payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-autores-b3108b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-b3108b

openshift-ingress

replicaset-controller

e2e-autores-b3108b-openshift-default-5d75b95f4b

SuccessfulCreate

Created pod: e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

openshift-ingress

GatewayAuthBootstrap

e2e-autores-b3108b

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-autores-b3108b-authn-ssl: envoyfilters.networking.istio.io "e2e-autores-b3108b-authn-ssl" already exists

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

Started

Started container payload-pre-processing

openshift-ingress

multus

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ait-85e90bf8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-85e90bf8

openshift-ingress

deployment-controller

e2e-ait-85e90bf8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-85e90bf8-openshift-default-6cddcfb646 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-85e90bf8-openshift-default-6cddcfb646

SuccessfulCreate

Created pod: e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

openshift-ingress

multus

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-b3108b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-b3108b-5c898cf69d from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-b3108b-5c898cf69d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-b3108b-5c898cf69d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-b3108b: serviceaccount "payload-processing-e2e-autores-b3108b" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-b3108b-9449f7f77

FailedCreate

Error creating: pods "payload-processing-e2e-autores-b3108b-9449f7f77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-b3108b: serviceaccount "payload-processing-e2e-autores-b3108b" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-b3108b-9449f7f77

SuccessfulCreate

Created pod: payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.596s (2.596s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

payload-processing-e2e-autores-b3108b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-b3108b-9449f7f77 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-b3108b-5c898cf69d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-ait-85e90bf8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-85e90bf8-7c44767c7c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-85e90bf8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-85e90bf8-5755c99898 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-85e90bf8-7c44767c7c

FailedCreate

Error creating: pods "payload-processing-e2e-ait-85e90bf8-7c44767c7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-85e90bf8: serviceaccount "payload-processing-e2e-ait-85e90bf8" not found

openshift-ingress

multus

payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-85e90bf8-7c44767c7c

SuccessfulCreate

Created pod: payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-85e90bf8-5755c99898

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-85e90bf8-5755c99898

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-85e90bf8-5755c99898-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-85e90bf8: serviceaccount "payload-processing-e2e-ait-85e90bf8" not found

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Unhealthy

Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

ProbeError

Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Unhealthy

Readiness probe failed: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

ProbeError

Readiness probe error: Get "http://10.133.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

deployment-controller

e2e-inf-b-4247c7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-4247c7-openshift-default-6c679cc99d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-b-4247c7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4247c7

openshift-ingress

replicaset-controller

e2e-inf-b-4247c7-openshift-default-6c679cc99d

SuccessfulCreate

Created pod: e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

openshift-ingress

multus

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-4247c7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-4247c7-869768d85c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-4247c7-869768d85c

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-4247c7-869768d85c

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-4247c7-869768d85c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4247c7: serviceaccount "payload-processing-e2e-inf-b-4247c7" not found
(x5)

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4247c7: serviceaccount "payload-processing-e2e-inf-b-4247c7" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-4247c7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-4247c7-675f8c4966 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

ProbeError

Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Unhealthy

Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-85e90bf8-5755c99898-75fjc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-85e90bf8-openshift-default-6cddcfb646-t6zzp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ait-85e90bf8-7c44767c7c-qqvgc

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576

SuccessfulCreate

Created pod: e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

openshift-ingress

deployment-controller

e2e-migrate-80c77821-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-80c77821-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-80c77821-gw
(x5)

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

ProbeError

Startup probe error: Get "http://10.132.0.37:15021/healthz/ready": dial tcp 10.132.0.37:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Unhealthy

Startup probe failed: Get "http://10.132.0.37:15021/healthz/ready": dial tcp 10.132.0.37:15021: connect: connection refused

openshift-ingress

multus

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-80c77821

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-80c77821-6679bf5b58 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-80c77821

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-80c77821-774f544854 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-migrate-80c77821-774f544854-wf28t

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-80c77821-774f544854

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-80c77821-774f544854-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-80c77821: serviceaccount "payload-processing-e2e-migrate-80c77821" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-80c77821: serviceaccount "payload-processing-e2e-migrate-80c77821" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-80c77821-774f544854

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-80c77821-774f544854-wf28t

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-80c77821-774f544854-wf28t

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-80c77821-774f544854-wf28t

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-80c77821-774f544854-wf28t

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Unhealthy

Readiness probe failed: Get "http://10.132.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

ProbeError

Readiness probe error: Get "http://10.132.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

ProbeError

Startup probe error: Get "http://10.132.0.38:15021/healthz/ready": dial tcp 10.132.0.38:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Unhealthy

Startup probe failed: Get "http://10.132.0.38:15021/healthz/ready": dial tcp 10.132.0.38:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-80c77821-6679bf5b58-v7qsq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-80c77821-774f544854-wf28t

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-migrate-80c77821-gw-openshift-default-6b5cc6f576-hz6cr

Killing

Stopping container istio-proxy

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-migrate-80c77821

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-migrate-80c77821: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-migrate-80c77821": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-migrate-80c77821, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 3b5f4b45-5724-4f6e-baa4-4b6f46134444, UID in object meta:

openshift-ingress

gateway_labeler_controller

e2e-ait-f0b1e2fe

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f0b1e2fe

openshift-ingress

replicaset-controller

e2e-ait-f0b1e2fe-openshift-default-887748fc4

SuccessfulCreate

Created pod: e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

openshift-ingress

deployment-controller

e2e-ait-f0b1e2fe-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-f0b1e2fe-openshift-default-887748fc4 from 0 to 1

openshift-ingress

multus

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-f0b1e2fe

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-f0b1e2fe

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5

FailedCreate

Error creating: pods "payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f0b1e2fe: serviceaccount "payload-processing-e2e-ait-f0b1e2fe" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5

SuccessfulCreate

Created pod: payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f0b1e2fe: serviceaccount "payload-processing-e2e-ait-f0b1e2fe" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

openshift-ingress

kubelet

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

ProbeError

Readiness probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Unhealthy

Readiness probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Unhealthy

Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

ProbeError

Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-f0b1e2fe

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-f0b1e2fe: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-f0b1e2fe": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-f0b1e2fe, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 66d407e3-a092-483f-b492-1b511afbda28, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-ait-f0b1e2fe-5cdb7488d5-bvgkn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f0b1e2fe-6f79c85786-62pww

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-f0b1e2fe-openshift-default-887748fc4-pk6ww

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-derive-588a50a1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-588a50a1

openshift-ingress

replicaset-controller

e2e-derive-588a50a1-openshift-default-55cd5fb984

SuccessfulCreate

Created pod: e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

openshift-ingress

deployment-controller

e2e-derive-588a50a1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-588a50a1-openshift-default-55cd5fb984 from 0 to 1

openshift-ingress

multus

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

openshift-ingress

deployment-controller

payload-processing-e2e-derive-588a50a1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-588a50a1-7d6fc5794f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-588a50a1-7d6fc5794f

SuccessfulCreate

Created pod: payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-588a50a1: serviceaccount "payload-processing-e2e-derive-588a50a1" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-588a50a1-7d6fc5794f

FailedCreate

Error creating: pods "payload-processing-e2e-derive-588a50a1-7d6fc5794f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-588a50a1: serviceaccount "payload-processing-e2e-derive-588a50a1" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-588a50a1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd from 0 to 1

openshift-ingress

multus

payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-b3108b-5c898cf69d-bqxcr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-b3108b-9449f7f77-z27gg

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-explicit-faed1b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-faed1b

openshift-ingress

deployment-controller

e2e-explicit-faed1b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-faed1b-openshift-default-644ddf89b4 from 0 to 1

openshift-ingress

kubelet

e2e-autores-b3108b-openshift-default-5d75b95f4b-9hh7d

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-explicit-faed1b-openshift-default-644ddf89b4

SuccessfulCreate

Created pod: e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

openshift-ingress

multus

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Unhealthy

Readiness probe failed: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

ProbeError

Readiness probe error: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-faed1b-78cff8c469

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-faed1b-78cff8c469-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-faed1b: serviceaccount "payload-processing-e2e-explicit-faed1b" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-faed1b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Started

Started container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-faed1b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-faed1b-78cff8c469 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-faed1b-78cff8c469

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-faed1b: serviceaccount "payload-processing-e2e-explicit-faed1b" not found
(x2)

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Unhealthy

Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

ProbeError

Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

Unhealthy

Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

ProbeError

Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-derive-588a50a1

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-derive-588a50a1: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-derive-588a50a1": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-derive-588a50a1, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 0341ee41-53ab-41bb-87a4-ef9e5a03acd3, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-derive-588a50a1-7d6fc5794f-5xs8d

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-588a50a1-5c585bb6fd-6tqtz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-588a50a1-openshift-default-55cd5fb984-dh9zq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-faed1b-6756ffcc7d-lvs7d

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-faed1b-78cff8c469-n6n44

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-explicit-faed1b-openshift-default-644ddf89b4-jsrj9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

multus

e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

e2e-noten-8d88a2-openshift-default-7dc5c55b98

SuccessfulCreate

Created pod: e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

openshift-ingress

gateway_labeler_controller

e2e-noten-8d88a2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-8d88a2

openshift-ingress

deployment-controller

e2e-noten-8d88a2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-8d88a2-openshift-default-7dc5c55b98 from 0 to 1

openshift-ingress

kubelet

e2e-noten-8d88a2-openshift-default-7dc5c55b98-7mqlx

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-mt-9d2e2715-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-9d2e2715-openshift-default-5dbb96d66d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-9d2e2715

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-9d2e2715

openshift-ingress

replicaset-controller

e2e-mt-9d2e2715-openshift-default-5dbb96d66d

SuccessfulCreate

Created pod: e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-8300b9-5474bb45f4-xv6f8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-8300b9-855c8cfdc7-wb9qf

Killing

Stopping container payload-processing

openshift-ingress

multus

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9d2e2715-77fc999f9c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-9d2e2715-77fc999f9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9d2e2715: serviceaccount "payload-processing-e2e-mt-9d2e2715" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-9d2e2715

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-9d2e2715-77fc999f9c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-9d2e2715

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9d2e2715: serviceaccount "payload-processing-e2e-mt-9d2e2715" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9d2e2715-77fc999f9c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

openshift-ingress

kubelet

payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-a-8300b9-openshift-default-6fd8579f5d-j7bzc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4247c7-675f8c4966-p2hnl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Unhealthy

Readiness probe failed: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

ProbeError

Readiness probe error: Get "http://10.132.0.48:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4247c7-869768d85c-54wcn

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-b-4247c7

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-b-4247c7: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-b-4247c7": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-b-4247c7, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 62da89f9-b768-4b46-bc62-2b27ee0fd5a3, UID in object meta:
(x2)

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

ProbeError

Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Unhealthy

Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8

SuccessfulCreate

Created pod: e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-b-4247c7-openshift-default-6c679cc99d-lk98v

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

e2e-shared-a-w3-14a8d1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-14a8d1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-14a8d1

openshift-ingress

multus

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

AddedInterface

Add eth0 [10.133.0.79/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-14a8d1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-14a8d1: serviceaccount "payload-processing-e2e-shared-a-w3-14a8d1" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-14a8d1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-14a8d1: serviceaccount "payload-processing-e2e-shared-a-w3-14a8d1" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Unhealthy

Readiness probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

ProbeError

Readiness probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-9d2e2715-77fc999f9c-4866g

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9d2e2715-dc85b9ff4-8thc9

Unhealthy

Readiness probe failed: dial tcp 10.134.0.48:9004: i/o timeout
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Unhealthy

Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

ProbeError

Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46

SuccessfulCreate

Created pod: e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-cab4ee

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-cab4ee

openshift-ingress

deployment-controller

e2e-shared-b-w3-cab4ee-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46 from 0 to 1

openshift-ingress

multus

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-9d2e2715-openshift-default-5dbb96d66d-t7967

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-cab4ee

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-cab4ee

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-cab4ee: serviceaccount "payload-processing-e2e-shared-b-w3-cab4ee" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-cab4ee: serviceaccount "payload-processing-e2e-shared-b-w3-cab4ee" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

Created

Created container: payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-8dd4a3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-8dd4a3

openshift-ingress

deployment-controller

e2e-shared-a-w2-8dd4a3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8

SuccessfulCreate

Created pod: e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

openshift-ingress

multus

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

ProbeError

Readiness probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Unhealthy

Readiness probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-8dd4a3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3: serviceaccount "payload-processing-e2e-shared-a-w2-8dd4a3" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-8dd4a3: serviceaccount "payload-processing-e2e-shared-a-w2-8dd4a3" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-8dd4a3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m

Started

Started container payload-processing
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

ProbeError

Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Unhealthy

Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-4bc8b3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-4bc8b3

openshift-ingress

multus

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

AddedInterface

Add eth0 [10.133.0.83/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976

SuccessfulCreate

Created pod: e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

openshift-ingress

deployment-controller

e2e-shared-b-w2-4bc8b3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-4bc8b3-openshift-default-845855d976 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-4bc8b3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-4bc8b3: serviceaccount "payload-processing-e2e-shared-b-w2-4bc8b3" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-4bc8b3: serviceaccount "payload-processing-e2e-shared-b-w2-4bc8b3" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-4bc8b3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Unhealthy

Readiness probe failed: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

ProbeError

Readiness probe error: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

ProbeError

Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Unhealthy

Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-mt-c615f98e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-c615f98e-openshift-default-7fbbb5c8d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-c615f98e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-c615f98e

openshift-ingress

GatewayAuthBootstrap

e2e-mt-c615f98e

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-mt-c615f98e-authn-ssl: envoyfilters.networking.istio.io "e2e-mt-c615f98e-authn-ssl" already exists

openshift-ingress

replicaset-controller

e2e-mt-c615f98e-openshift-default-7fbbb5c8d

SuccessfulCreate

Created pod: e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

openshift-ingress

multus

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-c615f98e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-c615f98e-795f74b95d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-c615f98e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-c615f98e-768d58678f from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-c615f98e-768d58678f-klqfn

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c615f98e-795f74b95d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-c615f98e-795f74b95d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c615f98e: serviceaccount "payload-processing-e2e-mt-c615f98e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c615f98e-795f74b95d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c615f98e-768d58678f

SuccessfulCreate

Created pod: payload-processing-e2e-mt-c615f98e-768d58678f-klqfn
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c615f98e-768d58678f

FailedCreate

Error creating: pods "payload-processing-e2e-mt-c615f98e-768d58678f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c615f98e: serviceaccount "payload-processing-e2e-mt-c615f98e" not found

openshift-ingress

multus

payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-c615f98e-768d58678f-klqfn

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-c615f98e-768d58678f-klqfn

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-c615f98e-768d58678f-klqfn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

ProbeError

Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

Unhealthy

Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-mt-c615f98e-768d58678f-klqfn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c615f98e-795f74b95d-vl86h

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-c615f98e-openshift-default-7fbbb5c8d-dggrv

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-api-a-439275-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-439275-openshift-default-666b8ddd88 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-439275-openshift-default-666b8ddd88

SuccessfulCreate

Created pod: e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

openshift-ingress

multus

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-api-a-439275

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-439275

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-439275-6d855c68cd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-439275

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-439275-5b68bd7bc9 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-439275-6d855c68cd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-439275-6d855c68cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-439275: serviceaccount "payload-processing-e2e-api-a-439275" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-439275-5b68bd7bc9

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-439275-5b68bd7bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-439275: serviceaccount "payload-processing-e2e-api-a-439275" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-439275-5b68bd7bc9

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-439275

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-439275-6d855c68cd from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Unhealthy

Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

ProbeError

Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-api-b-cbac1f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8 from 0 to 1

openshift-ingress

multus

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8

SuccessfulCreate

Created pod: e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

openshift-ingress

gateway_labeler_controller

e2e-api-b-cbac1f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-cbac1f
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cbac1f: serviceaccount "payload-processing-e2e-api-b-cbac1f" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-cbac1f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-cbac1f-f64964c6b from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-cbac1f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-cbac1f-b4d74585c from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-cbac1f-b4d74585c

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-cbac1f-b4d74585c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cbac1f: serviceaccount "payload-processing-e2e-api-b-cbac1f" not found

openshift-ingress

multus

payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

openshift-ingress

kubelet

payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-cbac1f-b4d74585c

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl
(x2)

openshift-ingress

kubelet

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-439275-6d855c68cd-vw4pn

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-439275-5b68bd7bc9-8rrjn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

Unhealthy

Readiness probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused

openshift-ingress

kubelet

e2e-api-a-439275-openshift-default-666b8ddd88-4jpmd

ProbeError

Readiness probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-b-cbac1f-b4d74585c-24dnl

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-b-cbac1f

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-b-cbac1f: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-b-cbac1f": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-b-cbac1f, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 7d1b48c4-a31f-4ad4-be80-579b6dbb0548, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-cbac1f-f64964c6b-h847c

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-b-cbac1f-openshift-default-5d4cdbcfc8-w88f7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-8dd4a3-75545f76bb-pmdbk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-8dd4a3-6ff9bb8949-mbb6m

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w2-8dd4a3-openshift-default-856bd4c5f8-24xmz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-4bc8b3-f47d97dbb-slvtb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-4bc8b3-6c8d9757c5-9rk5w

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w2-4bc8b3-openshift-default-845855d976-7ktqx

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-df863b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-df863b

openshift-ingress

multus

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ipp-a-df863b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-df863b-openshift-default-84d4c7f644 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-df863b-openshift-default-84d4c7f644

SuccessfulCreate

Created pod: e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-df863b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-df863b-79c6ff95c5 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-df863b-79c6ff95c5

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-df863b-79c6ff95c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-df863b: serviceaccount "payload-processing-e2e-ipp-a-df863b" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-df863b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-df863b-79948f674b from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-df863b-79948f674b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-df863b-79948f674b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-df863b: serviceaccount "payload-processing-e2e-ipp-a-df863b" not found

openshift-ingress

multus

payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-df863b-79948f674b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-df863b-79c6ff95c5

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

Started

Started container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Unhealthy

Readiness probe failed: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

ProbeError

Readiness probe error: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Unhealthy

Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

ProbeError

Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-ipp-b-3765cf-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-3765cf-openshift-default-5994cd68fb from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-3765cf

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-3765cf

openshift-ingress

replicaset-controller

e2e-ipp-b-3765cf-openshift-default-5994cd68fb

SuccessfulCreate

Created pod: e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

openshift-ingress

multus

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3765cf: serviceaccount "payload-processing-e2e-ipp-b-3765cf" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-3765cf

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-3765cf

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-3765cf-7d8f568c5 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-3765cf-7d8f568c5

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-3765cf-7d8f568c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3765cf: serviceaccount "payload-processing-e2e-ipp-b-3765cf" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-3765cf-7d8f568c5

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-8a4013

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-8a4013

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d

SuccessfulCreate

Created pod: e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

openshift-ingress

deployment-controller

e2e-ipp-cleanup-8a4013-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d from 0 to 1

openshift-ingress

multus

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-8a4013

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-8a4013

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013: serviceaccount "payload-processing-e2e-ipp-cleanup-8a4013" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-8a4013: serviceaccount "payload-processing-e2e-ipp-cleanup-8a4013" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Unhealthy

Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

ProbeError

Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

ProbeError

Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Unhealthy

Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8a4013-75776b58fc-qlxdv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8a4013-6fc85c8f4-76xmk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-ipp-cleanup-8a4013-openshift-default-c7f86645d-vq2l4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-df863b-79c6ff95c5-nstdw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-df863b-79948f674b-gfn6r

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-a-df863b-openshift-default-84d4c7f644-8wtmd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-3765cf-7d8f568c5-6r8j4

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-3765cf-6fb74cc7d-mq26q

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-b-3765cf-openshift-default-5994cd68fb-42qbw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-14a8d1-5fcc5bf5b5-vhxv6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-14a8d1-c55cb5658-dn7gw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-a-w3-14a8d1-openshift-default-6c866fcfc8-lrf89

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-cab4ee-5fc779cb6-f4nbr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-cab4ee-8448dc77cc-fpmmz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w3-cab4ee-openshift-default-945f9dd46-2dq66

Killing

Stopping container istio-proxy