Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

Scheduled

Successfully assigned openshift-ingress/e2e-derive-853daece-openshift-default-664896b9b-xkbg8 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-74969cc5f8-nzb2n

Scheduled

Successfully assigned openshift-ingress/payload-processing-74969cc5f8-nzb2n to ip-10-0-141-180.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-40c586-c868cd87-nfp55

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-40c586-c868cd87-nfp55 to ip-10-0-129-88.ec2.internal

openshift-ingress

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz to ip-10-0-129-88.ec2.internal

openshift-ingress

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9 to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx to ip-10-0-129-88.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-fnd68

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-fnd68 to ip-10-0-141-180.ec2.internal

openshift-ingress

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Scheduled

Successfully assigned openshift-ingress/e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4 to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx to ip-10-0-129-88.ec2.internal

openshift-ingress

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-cr47g

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-cr47g to ip-10-0-141-180.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4 to ip-10-0-129-88.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-cqtx5

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-cqtx5 to ip-10-0-141-180.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87 to ip-10-0-133-188.ec2.internal

openshift-ingress

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Scheduled

Successfully assigned openshift-ingress/e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz to ip-10-0-133-188.ec2.internal

openshift-ingress

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Scheduled

Successfully assigned openshift-ingress/e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz to ip-10-0-133-188.ec2.internal

openshift-ingress

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

Scheduled

Successfully assigned openshift-ingress/e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz to ip-10-0-133-188.ec2.internal

openshift-ingress

kube-auth-proxy-66f955bb57-bjdhw

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-66f955bb57-bjdhw to ip-10-0-129-88.ec2.internal

openshift-ingress

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl to ip-10-0-133-188.ec2.internal

openshift-ingress

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8 to ip-10-0-129-88.ec2.internal

openshift-ingress

kube-auth-proxy-66f955bb57-d45jp

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-66f955bb57-d45jp to ip-10-0-141-180.ec2.internal

openshift-ingress

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-40c586-openshift-default-68c58bff8d-phz44 to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-t5rpm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-t5rpm to ip-10-0-141-180.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7 to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8 to ip-10-0-129-88.ec2.internal

openshift-ingress

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-pre-processing-558d9b8847-hbpwz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-558d9b8847-hbpwz to ip-10-0-141-180.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7 to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9 to ip-10-0-141-180.ec2.internal

openshift-ingress

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb to ip-10-0-133-188.ec2.internal

openshift-ingress

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4 to ip-10-0-133-188.ec2.internal

openshift-ingress

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm to ip-10-0-129-88.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hdvvq to ip-10-0-133-188.ec2.internal

openshift-ingress

router-default-6d9b48664b-648rd

Scheduled

Successfully assigned openshift-ingress/router-default-6d9b48664b-648rd to ip-10-0-141-180.ec2.internal

openshift-ingress

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln to ip-10-0-129-88.ec2.internal

openshift-ingress

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22 to ip-10-0-129-88.ec2.internal

openshift-ingress

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f to ip-10-0-133-188.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v to ip-10-0-141-180.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-6d9b48664b from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x14)

openshift-ingress

replicaset-controller

router-default-6d9b48664b

FailedCreate

Error creating: pods "router-default-6d9b48664b-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-6d9b48664b

FailedCreate

Error creating: pods "router-default-6d9b48664b-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-6d9b48664b

SuccessfulCreate

Created pod: router-default-6d9b48664b-648rd

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

FailedMount

MountVolume.SetUp failed for volume "stats-auth" : failed to sync secret cache: timed out waiting for the condition

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

FailedMount

MountVolume.SetUp failed for volume "kube-api-access-rq949" : failed to sync configmap cache: timed out waiting for the condition

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : failed to sync configmap cache: timed out waiting for the condition
(x6)

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

Created

Created container: router

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

Started

Started container router

openshift-ingress

multus

router-default-6d9b48664b-648rd

AddedInterface

Add eth0 [10.133.0.10/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-6d9b48664b-648rd

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-66f955bb57 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-66f955bb57

SuccessfulCreate

Created pod: kube-auth-proxy-66f955bb57-d45jp
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-66f955bb57

FailedCreate

Error creating: pods "kube-auth-proxy-66f955bb57-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-66f955bb57

SuccessfulCreate

Created pod: kube-auth-proxy-66f955bb57-bjdhw

openshift-ingress

multus

kube-auth-proxy-66f955bb57-bjdhw

AddedInterface

Add eth0 [10.132.0.24/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-bjdhw

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-66f955bb57-d45jp

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-d45jp

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-cqtx5

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-d45jp

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-d45jp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.68s (2.68s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-bjdhw

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-bjdhw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.651s (2.651s including waiting). Image size: 265120132 bytes.

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-cqtx5

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-d45jp

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-66f955bb57-bjdhw

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.579s (2.579s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

Started

Started container discovery

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-z4c87

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.7s (2.7s including waiting). Image size: 181318953 bytes.

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-fnd68

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fnd68

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" already present on machine

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-fnd68

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fnd68

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fnd68

Started

Started container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-cqtx5

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-cqtx5

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-hdvvq

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-cr47g

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-t5rpm

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-t5rpm

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-t5rpm

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-cr47g

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-5b54b7fbb7-cr47g

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-cr47g

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.062s (1.062s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-t5rpm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-cr47g

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-cr47g

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-t5rpm

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.369s (1.369s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-t5rpm

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

ProbeError

Startup probe error: Get "http://10.134.0.25:15021/healthz/ready": dial tcp 10.134.0.25:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-hdvvq

Unhealthy

Startup probe failed: Get "http://10.134.0.25:15021/healthz/ready": dial tcp 10.134.0.25:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-cr47g

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-t5rpm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-nzb2n

Pulling

Pulling image "quay.io/maas/odh-praxis-extproc:pr699-76cb977"

openshift-ingress

multus

payload-processing-74969cc5f8-nzb2n

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-558d9b8847 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-558d9b8847

SuccessfulCreate

Created pod: payload-pre-processing-558d9b8847-hbpwz

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-74969cc5f8 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-hbpwz

Pulling

Pulling image "quay.io/maas/odh-praxis-extproc:pr699-76cb977"

openshift-ingress

replicaset-controller

payload-processing-74969cc5f8

SuccessfulCreate

Created pod: payload-processing-74969cc5f8-nzb2n

openshift-ingress

multus

payload-pre-processing-558d9b8847-hbpwz

AddedInterface

Add eth0 [10.133.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-hbpwz

Pulled

Successfully pulled image "quay.io/maas/odh-praxis-extproc:pr699-76cb977" in 2.089s (2.089s including waiting). Image size: 120164829 bytes.

openshift-ingress

kubelet

payload-processing-74969cc5f8-nzb2n

Pulled

Successfully pulled image "quay.io/maas/odh-praxis-extproc:pr699-76cb977" in 2.122s (2.122s including waiting). Image size: 120164829 bytes.

openshift-ingress

kubelet

payload-processing-74969cc5f8-nzb2n

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-nzb2n

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-hbpwz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-hbpwz

Created

Created container: payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-inf-a-e467de

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-e467de

openshift-ingress

multus

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-a-e467de-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-e467de-openshift-default-5d494b94fb from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-a-e467de-openshift-default-5d494b94fb

SuccessfulCreate

Created pod: e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e467de: serviceaccount "payload-processing-e2e-inf-a-e467de" not found

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-e467de

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-e467de-5bdbf8f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-e467de-5bdbf8f9

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-e467de-5bdbf8f9

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-e467de-5bdbf8f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e467de: serviceaccount "payload-processing-e2e-inf-a-e467de" not found

openshift-ingress

multus

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-e467de

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.25s (1.25s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

Started

Started container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-c4b97c19

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c4b97c19

openshift-ingress

deployment-controller

e2e-ait-c4b97c19-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-c4b97c19-openshift-default-7776d79bd8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-c4b97c19-openshift-default-7776d79bd8

SuccessfulCreate

Created pod: e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

openshift-ingress

multus

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-c4b97c19

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-c4b97c19-7fdf86c6d7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7

FailedCreate

Error creating: pods "payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c4b97c19: serviceaccount "payload-processing-e2e-ait-c4b97c19" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-c4b97c19

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

openshift-ingress

kubelet

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7

SuccessfulCreate

Created pod: payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c4b97c19: serviceaccount "payload-processing-e2e-ait-c4b97c19" not found

openshift-ingress

replicaset-controller

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d

SuccessfulCreate

Created pod: e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

openshift-ingress

gateway_labeler_controller

e2e-inf-b-bc1117

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-bc1117

openshift-ingress

deployment-controller

e2e-inf-b-bc1117-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d from 0 to 1

openshift-ingress

multus

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Started

Started container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-bc1117

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-bc1117-5ffb4f5889 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-bc1117-5ffb4f5889

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-bc1117-5ffb4f5889-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-bc1117: serviceaccount "payload-processing-e2e-inf-b-bc1117" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-bc1117: serviceaccount "payload-processing-e2e-inf-b-bc1117" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-bc1117

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-bc1117-5ffb4f5889

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

openshift-ingress

kubelet

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

Created

Created container: payload-processing
(x6)

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Unhealthy

Startup probe failed: Get "http://10.134.0.29:15021/healthz/ready": dial tcp 10.134.0.29:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

ProbeError

Startup probe error: Get "http://10.134.0.29:15021/healthz/ready": dial tcp 10.134.0.29:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Unhealthy

Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

ProbeError

Startup probe error: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Unhealthy

Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

ProbeError

Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c4b97c19-77ccddfbc4-92kjd

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-c4b97c19

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-c4b97c19: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-c4b97c19": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-c4b97c19, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6c31b91b-d778-4332-9d3e-3a116cef6f71, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-ait-c4b97c19-7fdf86c6d7-h4wdx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-migrate-07ec96b6-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb from 0 to 1

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-ait-c4b97c19-openshift-default-7776d79bd8-bjqgz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

gateway_labeler_controller

e2e-migrate-07ec96b6-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-07ec96b6-gw

openshift-ingress

replicaset-controller

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb

SuccessfulCreate

Created pod: e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

openshift-ingress

multus

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-07ec96b6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-07ec96b6-9568f45db from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-07ec96b6: serviceaccount "payload-processing-e2e-migrate-07ec96b6" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-07ec96b6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-07ec96b6-8489d8db9d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-07ec96b6-8489d8db9d

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-07ec96b6-8489d8db9d

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-07ec96b6-8489d8db9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-07ec96b6: serviceaccount "payload-processing-e2e-migrate-07ec96b6" not found

openshift-ingress

kubelet

payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

ProbeError

Readiness probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Unhealthy

Readiness probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Unhealthy

Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

ProbeError

Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-07ec96b6-9568f45db-jls8d

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-07ec96b6-8489d8db9d-tqgf8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-migrate-07ec96b6-gw-openshift-default-686c98dbfb-v5dmb

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-89bb5623-openshift-default-7b9df9846b

SuccessfulCreate

Created pod: e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ait-89bb5623

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-89bb5623

openshift-ingress

deployment-controller

e2e-ait-89bb5623-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-89bb5623-openshift-default-7b9df9846b from 0 to 1

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 3.085s (3.086s including waiting). Image size: 181318953 bytes.

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-89bb5623-7858c64ff7

SuccessfulCreate

Created pod: payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-89bb5623: serviceaccount "payload-processing-e2e-ait-89bb5623" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-89bb5623

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-89bb5623-ccb5678c8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

openshift-ingress

deployment-controller

payload-processing-e2e-ait-89bb5623

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-89bb5623-7858c64ff7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-89bb5623-7858c64ff7

FailedCreate

Error creating: pods "payload-processing-e2e-ait-89bb5623-7858c64ff7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-89bb5623: serviceaccount "payload-processing-e2e-ait-89bb5623" not found

openshift-ingress

kubelet

payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

ProbeError

Readiness probe error: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Unhealthy

Readiness probe failed: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

ProbeError

Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Unhealthy

Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ait-89bb5623-7858c64ff7-74nr8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-89bb5623-ccb5678c8-r85c7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-89bb5623-openshift-default-7b9df9846b-6g8hx

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-derive-853daece-openshift-default-664896b9b

SuccessfulCreate

Created pod: e2e-derive-853daece-openshift-default-664896b9b-xkbg8

openshift-ingress

gateway_labeler_controller

e2e-derive-853daece

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-853daece

openshift-ingress

deployment-controller

e2e-derive-853daece-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-853daece-openshift-default-664896b9b from 0 to 1

openshift-ingress

multus

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-853daece-647bf6f75c

SuccessfulCreate

Created pod: payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

openshift-ingress

multus

payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-853daece-94f67c57d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-853daece-94f67c57d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-853daece: serviceaccount "payload-processing-e2e-derive-853daece" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-853daece-94f67c57d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-853daece-647bf6f75c

FailedCreate

Error creating: pods "payload-processing-e2e-derive-853daece-647bf6f75c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-853daece: serviceaccount "payload-processing-e2e-derive-853daece" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-853daece

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-853daece-94f67c57d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-derive-853daece

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-853daece-647bf6f75c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e467de-776cc9c5f8-8mn6v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e467de-5bdbf8f9-m9pvx

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

ProbeError

Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

Unhealthy

Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-derive-853daece-647bf6f75c-tqbnt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-853daece-94f67c57d-z75lk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-853daece-openshift-default-664896b9b-xkbg8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-a-e467de-openshift-default-5d494b94fb-n4fj4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

Unhealthy

Readiness probe failed: dial tcp 10.132.0.39:9004: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-inf-b-bc1117-5ffb4f5889-z4lq4

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

Unhealthy

Readiness probe failed: dial tcp 10.132.0.38:9004: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-bc1117-84fd687bc8-zbz47

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8

SuccessfulCreate

Created pod: e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-0c919e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-0c919e

openshift-ingress

deployment-controller

e2e-shared-a-w2-0c919e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8 from 0 to 1

openshift-ingress

multus

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-b-bc1117-openshift-default-6ccfb4fb9d-qrc8f

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-0c919e: serviceaccount "payload-processing-e2e-shared-a-w2-0c919e" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-0c919e: serviceaccount "payload-processing-e2e-shared-a-w2-0c919e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-0c919e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-0c919e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

ProbeError

Readiness probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Unhealthy

Readiness probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4

SuccessfulCreate

Created pod: e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

openshift-ingress

deployment-controller

e2e-shared-b-w2-f83c87-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-f83c87-openshift-default-58d58454f4 from 0 to 1

openshift-ingress

multus

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-f83c87

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-f83c87

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-f83c87-84986b885

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-f83c87

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-f83c87

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-f83c87-84986b885 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f83c87: serviceaccount "payload-processing-e2e-shared-b-w2-f83c87" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-f83c87-84986b885

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-f83c87-84986b885-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f83c87: serviceaccount "payload-processing-e2e-shared-b-w2-f83c87" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Unhealthy

Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": read tcp 10.134.0.2:56104->10.134.0.41:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

ProbeError

Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": read tcp 10.134.0.2:56104->10.134.0.41:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:

openshift-ingress

GatewayAuthBootstrap

e2e-mt-dfbaeab8

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-mt-dfbaeab8-authn-ssl: envoyfilters.networking.istio.io "e2e-mt-dfbaeab8-authn-ssl" already exists

openshift-ingress

gateway_labeler_controller

e2e-mt-dfbaeab8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-dfbaeab8

openshift-ingress

deployment-controller

e2e-mt-dfbaeab8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-dfbaeab8-openshift-default-5749f8dc6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-dfbaeab8-openshift-default-5749f8dc6

SuccessfulCreate

Created pod: e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

openshift-ingress

multus

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-dfbaeab8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7

SuccessfulCreate

Created pod: payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-dfbaeab8: serviceaccount "payload-processing-e2e-mt-dfbaeab8" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

openshift-ingress

deployment-controller

payload-processing-e2e-mt-dfbaeab8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-dfbaeab8-5c4c7998f7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7

FailedCreate

Error creating: pods "payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-dfbaeab8: serviceaccount "payload-processing-e2e-mt-dfbaeab8" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-mt-dfbaeab8-5c4c7998f7-2pj22

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-dfbaeab8-74fdf89f66-bcld6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-mt-dfbaeab8-openshift-default-5749f8dc6-xntfd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w1-6e87d0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-6e87d0

openshift-ingress

deployment-controller

e2e-shared-a-w1-6e87d0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6 from 0 to 1

openshift-ingress

multus

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6

SuccessfulCreate

Created pod: e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-6e87d0: serviceaccount "payload-processing-e2e-shared-a-w1-6e87d0" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-6e87d0: serviceaccount "payload-processing-e2e-shared-a-w1-6e87d0" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w1-6e87d0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w1-6e87d0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

Unhealthy

Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

ProbeError

Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6

SuccessfulCreate

Created pod: e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w1-9f77a9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-9f77a9

openshift-ingress

deployment-controller

e2e-shared-b-w1-9f77a9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6 from 0 to 1

openshift-ingress

GatewayAuthBootstrap

e2e-shared-b-w1-9f77a9

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-shared-b-w1-9f77a9-authn-ssl: envoyfilters.networking.istio.io "e2e-shared-b-w1-9f77a9-authn-ssl" already exists

openshift-ingress

multus

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9: serviceaccount "payload-processing-e2e-shared-b-w1-9f77a9" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

AddedInterface

Add eth0 [10.133.0.73/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w1-9f77a9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n

Created

Created container: payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w1-9f77a9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-9f77a9: serviceaccount "payload-processing-e2e-shared-b-w1-9f77a9" not found
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Unhealthy

Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

ProbeError

Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-mt-d6ac6dc8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d6ac6dc8

openshift-ingress

deployment-controller

e2e-mt-d6ac6dc8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84

SuccessfulCreate

Created pod: e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

openshift-ingress

multus

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d6ac6dc8-767c678569

FailedCreate

Error creating: pods "payload-processing-e2e-mt-d6ac6dc8-767c678569-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d6ac6dc8: serviceaccount "payload-processing-e2e-mt-d6ac6dc8" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-d6ac6dc8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-d6ac6dc8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-d6ac6dc8-767c678569 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d6ac6dc8: serviceaccount "payload-processing-e2e-mt-d6ac6dc8" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d6ac6dc8-767c678569

SuccessfulCreate

Created pod: payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

openshift-ingress

multus

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-mt-d6ac6dc8-767c678569-dt4ln

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d6ac6dc8-7c7f565974-gpnr9

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-d6ac6dc8

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-d6ac6dc8: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-d6ac6dc8": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-d6ac6dc8, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e8f6c6f1-0d98-41dc-878a-2056ee01c106, UID in object meta:

openshift-ingress

kubelet

e2e-mt-d6ac6dc8-openshift-default-78dd8f8c84-dt9fz

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-2611af

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-2611af

openshift-ingress

deployment-controller

e2e-ipp-a-2611af-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-2611af-openshift-default-7566688f68 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-2611af-openshift-default-7566688f68

SuccessfulCreate

Created pod: e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

openshift-ingress

multus

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2611af-76c9b5d55f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-2611af-76c9b5d55f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2611af: serviceaccount "payload-processing-e2e-ipp-a-2611af" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

openshift-ingress

multus

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-a-40c586-openshift-default-68c58bff8d

SuccessfulCreate

Created pod: e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

openshift-ingress

deployment-controller

e2e-api-a-40c586-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-40c586-openshift-default-68c58bff8d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-a-40c586

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-40c586

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-2611af

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-2611af

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-2611af-76c9b5d55f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-2611af-76c9b5d55f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2611af: serviceaccount "payload-processing-e2e-ipp-a-2611af" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-40c586

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-40c586-fd69988f8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-40c586

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-40c586-c868cd87 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-40c586-fd69988f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-40c586-c868cd87

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-40c586-c868cd87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-40c586: serviceaccount "payload-processing-e2e-api-a-40c586" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-40c586-fd69988f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-40c586-fd69988f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-40c586: serviceaccount "payload-processing-e2e-api-a-40c586" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-40c586-c868cd87

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-40c586-c868cd87-nfp55

openshift-ingress

kubelet

payload-processing-e2e-api-a-40c586-c868cd87-nfp55

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-40c586-c868cd87-nfp55

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-a-40c586-c868cd87-nfp55

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-40c586-c868cd87-nfp55

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.452s (1.452s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

deployment-controller

e2e-ipp-b-bdf5b3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6

SuccessfulCreate

Created pod: e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-bdf5b3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-bdf5b3

openshift-ingress

multus

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-api-b-13d31c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-13d31c-openshift-default-8555c9955b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-13d31c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-13d31c

openshift-ingress

replicaset-controller

e2e-api-b-13d31c-openshift-default-8555c9955b

SuccessfulCreate

Created pod: e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

openshift-ingress

multus

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

ProbeError

Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Unhealthy

Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3: serviceaccount "payload-processing-e2e-ipp-b-bdf5b3" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-bdf5b3: serviceaccount "payload-processing-e2e-ipp-b-bdf5b3" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-bdf5b3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-bdf5b3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-bdf5b3-ddd87df55 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

Started

Started container payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-13d31c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-13d31c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-13d31c-77d8cd56ff from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-13d31c: serviceaccount "payload-processing-e2e-api-b-13d31c" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-13d31c-77d8cd56ff

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-13d31c-77d8cd56ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-13d31c: serviceaccount "payload-processing-e2e-api-b-13d31c" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-13d31c-77d8cd56ff

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

openshift-ingress

kubelet

payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

openshift-ingress

multus

payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Unhealthy

Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

ProbeError

Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-40c586-c868cd87-nfp55

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-40c586-fd69988f8-mnkm7

Unhealthy

Readiness probe failed: dial tcp 10.134.0.67:9004: i/o timeout

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-a-40c586-openshift-default-68c58bff8d-phz44

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-13d31c-79d5fc8bdb-898cb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-13d31c-77d8cd56ff-8w629

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-13d31c-openshift-default-8555c9955b-ps8wb

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8

SuccessfulCreate

Created pod: e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

openshift-ingress

deployment-controller

e2e-ipp-cleanup-a35f4b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-a35f4b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-a35f4b

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-6e87d0-7f7bfc4cf8-hk69m

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-a35f4b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b: serviceaccount "payload-processing-e2e-ipp-cleanup-a35f4b" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-6e87d0-6ddb98b6f5-kgqzm

Killing

Stopping container payload-processing

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-a35f4b: serviceaccount "payload-processing-e2e-ipp-cleanup-a35f4b" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-a35f4b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w1-6e87d0-openshift-default-85bb475c6-hvltq

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-9f77a9-7cb54c9b7d-tlh6n

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w1-9f77a9

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w1-9f77a9: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w1-9f77a9": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w1-9f77a9, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4fb49062-8c3a-4f6b-b535-b63630dcdeef, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-9f77a9-6b8cd96b59-grmh9

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-a35f4b-6657d7c56f-zf8cz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-a35f4b-555d895794-7knt2

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w1-9f77a9-openshift-default-77597cb9c6-gvgbl

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ipp-cleanup-a35f4b-openshift-default-7d75dbcbf8-s64nf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-2611af-76c9b5d55f-rh4n9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-2611af-7cdddf7dc8-dffp9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-2611af-openshift-default-7566688f68-48f2z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-bdf5b3-ddd87df55-zrwjz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-bdf5b3-6496844874-bm8z4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-b-bdf5b3-openshift-default-84ffbd9bf6-vbj2s

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-0c919e-54c556dddb-c9qxb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-0c919e-647b5bf5ff-l54lh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w2-0c919e-openshift-default-7d69f7c5b8-rw8rz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f83c87-84986b885-tjvp7

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w2-f83c87

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f83c87: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w2-f83c87": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f83c87, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a1808cf1-a482-4b04-b639-31eceb7d49c0, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f83c87-dbcfc978b-4bj67

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w2-f83c87-openshift-default-58d58454f4-fvlf9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x19)

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-hbpwz

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (payload-processing-dockercfg-zhr5x); attempting to pull the image may not succeed.
(x20)

openshift-ingress

kubelet

payload-processing-74969cc5f8-nzb2n

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (payload-processing-dockercfg-zhr5x); attempting to pull the image may not succeed.