Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w to ip-10-0-129-1.ec2.internal

openshift-ingress

router-default-6bf9477994-mzx8r

FailedScheduling

0/3 nodes are available: 3 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/3 nodes are available: 3 Preemption is not helpful for scheduling.

openshift-ingress

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6 to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5 to ip-10-0-129-1.ec2.internal

openshift-ingress

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Scheduled

Successfully assigned openshift-ingress/e2e-ait-34f15601-openshift-default-f98d86864-69bqr to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-processing-5cc6cd444b-pg5l2

Scheduled

Successfully assigned openshift-ingress/payload-processing-5cc6cd444b-pg5l2 to ip-10-0-129-1.ec2.internal

openshift-ingress

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh to ip-10-0-129-1.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-p6zbq to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6 to ip-10-0-129-1.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-687dbb6cf6-lfmh8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-687dbb6cf6-lfmh8 to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-7db38b79-97f947bd-brpq6 to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49 to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Scheduled

Successfully assigned openshift-ingress/e2e-mt-6958d802-openshift-default-7c46d7c684-td22h to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6 to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657 to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Scheduled

Successfully assigned openshift-ingress/e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk to ip-10-0-129-1.ec2.internal

openshift-ingress

kube-auth-proxy-96bfd8458-fkkfp

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-96bfd8458-fkkfp to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-62afff-555987f44b-fhchj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-62afff-555987f44b-fhchj to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f to ip-10-0-129-1.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-kdhht

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-kdhht to ip-10-0-128-47.ec2.internal

openshift-ingress

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Scheduled

Successfully assigned openshift-ingress/e2e-mt-f3708437-openshift-default-548f994dbd-74zs8 to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8 to ip-10-0-128-47.ec2.internal

openshift-ingress

kube-auth-proxy-96bfd8458-l6tqk

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-96bfd8458-l6tqk to ip-10-0-129-11.ec2.internal

openshift-ingress

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5 to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4 to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x to ip-10-0-129-11.ec2.internal

openshift-ingress

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Scheduled

Successfully assigned openshift-ingress/e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7 to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-55901c-596775948d-hp756

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-55901c-596775948d-hp756 to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756 to ip-10-0-128-47.ec2.internal

openshift-ingress

router-default-6bf9477994-mzx8r

Scheduled

Successfully assigned openshift-ingress/router-default-6bf9477994-mzx8r to ip-10-0-129-11.ec2.internal

openshift-ingress

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz to ip-10-0-129-11.ec2.internal

openshift-ingress

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489 to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js to ip-10-0-129-11.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx to ip-10-0-128-47.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h to ip-10-0-128-47.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-npxlj

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-npxlj to ip-10-0-129-1.ec2.internal

openshift-ingress

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n to ip-10-0-129-1.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s to ip-10-0-128-47.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-6bf9477994 from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x13)

openshift-ingress

replicaset-controller

router-default-6bf9477994

FailedCreate

Error creating: pods "router-default-6bf9477994-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-6bf9477994

FailedCreate

Error creating: pods "router-default-6bf9477994-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-6bf9477994

SuccessfulCreate

Created pod: router-default-6bf9477994-mzx8r
(x6)

openshift-ingress

kubelet

router-default-6bf9477994-mzx8r

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x6)

openshift-ingress

kubelet

router-default-6bf9477994-mzx8r

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-6bf9477994-mzx8r

Created

Created container: router

openshift-ingress

multus

router-default-6bf9477994-mzx8r

AddedInterface

Add eth0 [10.132.0.18/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-6bf9477994-mzx8r

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:627e54de31981702dfc5986fda8e14dd7490cc3ccfa94709d1b412defbb073bf" already present on machine

openshift-ingress

kubelet

router-default-6bf9477994-mzx8r

Started

Started container router

openshift-ingress

replicaset-controller

kube-auth-proxy-96bfd8458

SuccessfulCreate

Created pod: kube-auth-proxy-96bfd8458-l6tqk

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-l6tqk

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-96bfd8458

FailedCreate

Error creating: pods "kube-auth-proxy-96bfd8458-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-fkkfp

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-96bfd8458

SuccessfulCreate

Created pod: kube-auth-proxy-96bfd8458-fkkfp

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-96bfd8458 from 0 to 2

openshift-ingress

multus

kube-auth-proxy-96bfd8458-l6tqk

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-fkkfp

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-l6tqk

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-npxlj

openshift-ingress

multus

kube-auth-proxy-96bfd8458-fkkfp

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-npxlj

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-l6tqk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.808s (2.808s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-l6tqk

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-l6tqk

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-fkkfp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 6.827s (6.827s including waiting). Image size: 265120132 bytes.

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-kdhht

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-kdhht

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-kdhht

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-fkkfp

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Started

Started container discovery

openshift-ingress

kubelet

kube-auth-proxy-96bfd8458-fkkfp

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 5.922s (5.922s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-kdhht

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.077s (3.077s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-kdhht

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-kdhht

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

ProbeError

Readiness probe error: Get "http://10.134.0.23:8080/ready": dial tcp 10.134.0.23:8080: connect: connection refused body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-npxlj

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-npxlj

Unhealthy

Readiness probe failed: Get "http://10.134.0.23:8080/ready": dial tcp 10.134.0.23:8080: connect: connection refused

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-ghkbf

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 16.74s (16.74s including waiting). Image size: 181318953 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-p6zbq

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-687dbb6cf6 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-5cc6cd444b

FailedCreate

Error creating: pods "payload-processing-5cc6cd444b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5cc6cd444b

SuccessfulCreate

Created pod: payload-processing-5cc6cd444b-pg5l2

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5cc6cd444b from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-687dbb6cf6

SuccessfulCreate

Created pod: payload-pre-processing-687dbb6cf6-lfmh8
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-687dbb6cf6

FailedCreate

Error creating: pods "payload-pre-processing-687dbb6cf6-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-processing-5cc6cd444b-pg5l2

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-lfmh8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-5cc6cd444b-pg5l2

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-687dbb6cf6-lfmh8

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5cc6cd444b-pg5l2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5cc6cd444b-pg5l2

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.051s (2.051s including waiting). Image size: 165907127 bytes.

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-lfmh8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.106s (2.106s including waiting). Image size: 165907127 bytes.

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-lfmh8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-lfmh8

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-5cc6cd444b-pg5l2

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-lfmh8

Unhealthy

Readiness probe failed: dial tcp 10.134.0.42:9004: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

Unhealthy

Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-p6zbq

ProbeError

Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-ait-7db38b79-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-7db38b79-openshift-default-7ffd568bd8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-7db38b79-openshift-default-7ffd568bd8

SuccessfulCreate

Created pod: e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

openshift-ingress

gateway_labeler_controller

e2e-ait-7db38b79

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-7db38b79

openshift-ingress

multus

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-7db38b79-78c69598dd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-7db38b79-97f947bd

FailedCreate

Error creating: pods "payload-processing-e2e-ait-7db38b79-97f947bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7db38b79: serviceaccount "payload-processing-e2e-ait-7db38b79" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-7db38b79

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-7db38b79-78c69598dd from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-7db38b79

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-7db38b79-97f947bd from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-7db38b79-97f947bd

SuccessfulCreate

Created pod: payload-processing-e2e-ait-7db38b79-97f947bd-brpq6
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-7db38b79-78c69598dd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-7db38b79-78c69598dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-7db38b79: serviceaccount "payload-processing-e2e-ait-7db38b79" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.796s (1.796s including waiting). Image size: 165907127 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.389s (2.389s including waiting). Image size: 165907127 bytes.

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Unhealthy

Readiness probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

ProbeError

Readiness probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

ProbeError

Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Unhealthy

Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-7db38b79-78c69598dd-tbqj8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-7db38b79-97f947bd-brpq6

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-migrate-b6e21d41-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88 from 0 to 1

openshift-ingress

replicaset-controller

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88

SuccessfulCreate

Created pod: e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

openshift-ingress

gateway_labeler_controller

e2e-migrate-b6e21d41-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-b6e21d41-gw

openshift-ingress

multus

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-b6e21d41: serviceaccount "payload-processing-e2e-migrate-b6e21d41" not found

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-b6e21d41

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-b6e21d41-68fc5f78d5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-b6e21d41: serviceaccount "payload-processing-e2e-migrate-b6e21d41" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-b6e21d41

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ait-7db38b79-openshift-default-7ffd568bd8-tjjdk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

ProbeError

Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Unhealthy

Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6e21d41-7b4b485cb8-xc657

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6e21d41-68fc5f78d5-rj2qz

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-ait-34f15601-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-34f15601-openshift-default-f98d86864 from 0 to 1

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-34f15601-openshift-default-f98d86864

SuccessfulCreate

Created pod: e2e-ait-34f15601-openshift-default-f98d86864-69bqr

openshift-ingress

gateway_labeler_controller

e2e-ait-34f15601

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-34f15601

openshift-ingress

multus

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-migrate-b6e21d41-gw-openshift-default-c7f667c88-w4wht

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-processing-e2e-ait-34f15601

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-34f15601-6bbcd584f6 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-34f15601-6bbcd584f6

FailedCreate

Error creating: pods "payload-processing-e2e-ait-34f15601-6bbcd584f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-34f15601: serviceaccount "payload-processing-e2e-ait-34f15601" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-34f15601

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-34f15601-7b589bb45b from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-34f15601-6bbcd584f6

SuccessfulCreate

Created pod: payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

openshift-ingress

kubelet

payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-34f15601-7b589bb45b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

openshift-ingress

multus

payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-34f15601-7b589bb45b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-34f15601-7b589bb45b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-34f15601: serviceaccount "payload-processing-e2e-ait-34f15601" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Unhealthy

Readiness probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

ProbeError

Readiness probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Unhealthy

Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

ProbeError

Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-34f15601-7b589bb45b-76fs4

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-34f15601

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-34f15601: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-34f15601": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-34f15601, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8dde0730-8b66-47ec-b3cd-6b68f4488ab8, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-ait-34f15601-6bbcd584f6-4c5pl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-derive-b9abcfc3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-b9abcfc3

openshift-ingress

deployment-controller

e2e-derive-b9abcfc3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-b9abcfc3-openshift-default-5f46d9455b from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-b9abcfc3-openshift-default-5f46d9455b

SuccessfulCreate

Created pod: e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

openshift-ingress

multus

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ait-34f15601-openshift-default-f98d86864-69bqr

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-b9abcfc3-68b857578b

SuccessfulCreate

Created pod: payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-b9abcfc3-68b857578b

FailedCreate

Error creating: pods "payload-processing-e2e-derive-b9abcfc3-68b857578b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-b9abcfc3: serviceaccount "payload-processing-e2e-derive-b9abcfc3" not found

openshift-ingress

kubelet

payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-b9abcfc3: serviceaccount "payload-processing-e2e-derive-b9abcfc3" not found

openshift-ingress

deployment-controller

payload-processing-e2e-derive-b9abcfc3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-b9abcfc3-68b857578b from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-b9abcfc3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648 from 0 to 1

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

ProbeError

Readiness probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Unhealthy

Readiness probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Unhealthy

Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

ProbeError

Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-b9abcfc3-7cb8898648-9gq6m

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-b9abcfc3-68b857578b-jq29k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-derive-b9abcfc3-openshift-default-5f46d9455b-rp2t7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

multus

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-mt-6958d802

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-6958d802

openshift-ingress

deployment-controller

e2e-mt-6958d802-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-6958d802-openshift-default-7c46d7c684 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-6958d802-openshift-default-7c46d7c684

SuccessfulCreate

Created pod: e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-6958d802

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-6958d802-5648b8d788 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-6958d802-f66cbf59d

FailedCreate

Error creating: pods "payload-processing-e2e-mt-6958d802-f66cbf59d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6958d802: serviceaccount "payload-processing-e2e-mt-6958d802" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-6958d802

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-6958d802-f66cbf59d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-6958d802-f66cbf59d

SuccessfulCreate

Created pod: payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

openshift-ingress

kubelet

payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-6958d802-5648b8d788

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

openshift-ingress

multus

payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-6958d802-5648b8d788

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-6958d802-5648b8d788-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-6958d802: serviceaccount "payload-processing-e2e-mt-6958d802" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

Unhealthy

Readiness probe failed: dial tcp 10.133.0.32:9004: connect: connection refused

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Unhealthy

Readiness probe failed: Get "http://10.134.0.51:15021/healthz/ready": read tcp 10.134.0.2:33404->10.134.0.51:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

ProbeError

Readiness probe error: Get "http://10.134.0.51:15021/healthz/ready": read tcp 10.134.0.2:33404->10.134.0.51:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Unhealthy

Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

ProbeError

Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-6958d802-5648b8d788-94r8s

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-6958d802-f66cbf59d-cp4js

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-6958d802-openshift-default-7c46d7c684-td22h

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

gateway_labeler_controller

e2e-mt-f3708437

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-f3708437

openshift-ingress

replicaset-controller

e2e-mt-f3708437-openshift-default-548f994dbd

SuccessfulCreate

Created pod: e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

openshift-ingress

deployment-controller

e2e-mt-f3708437-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-f3708437-openshift-default-548f994dbd from 0 to 1

openshift-ingress

multus

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-f3708437

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-f3708437-55b87cdc9d from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-f3708437-64446c49b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-f3708437-64446c49b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f3708437: serviceaccount "payload-processing-e2e-mt-f3708437" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-f3708437-55b87cdc9d

FailedCreate

Error creating: pods "payload-processing-e2e-mt-f3708437-55b87cdc9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f3708437: serviceaccount "payload-processing-e2e-mt-f3708437" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-f3708437

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-f3708437-64446c49b from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-f3708437-64446c49b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-f3708437-55b87cdc9d

SuccessfulCreate

Created pod: payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Unhealthy

Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": read tcp 10.134.0.2:35432->10.134.0.52:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

ProbeError

Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": read tcp 10.134.0.2:35432->10.134.0.52:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-f3708437

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-f3708437: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-f3708437": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-f3708437, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: bd6bc9a0-e0cb-4a83-9e9f-d211aeb65e6b, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-mt-f3708437-55b87cdc9d-vnmqt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f3708437-64446c49b-np6gx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-f3708437-openshift-default-548f994dbd-74zs8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

e2e-inf-a-55901c-openshift-default-5466b565dd

SuccessfulCreate

Created pod: e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

openshift-ingress

gateway_labeler_controller

e2e-inf-a-55901c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-55901c

openshift-ingress

deployment-controller

e2e-inf-a-55901c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-55901c-openshift-default-5466b565dd from 0 to 1

openshift-ingress

multus

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-55901c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-55901c-596775948d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-55901c-596775948d

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-55901c-596775948d-hp756
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-55901c-596775948d

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-55901c-596775948d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-55901c: serviceaccount "payload-processing-e2e-inf-a-55901c" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-55901c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-55901c-86895d48c6 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-55901c-86895d48c6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-55901c-86895d48c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-55901c: serviceaccount "payload-processing-e2e-inf-a-55901c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-55901c-86895d48c6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-55901c-596775948d-hp756

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-55901c-596775948d-hp756

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-a-55901c-596775948d-hp756

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-inf-a-55901c-596775948d-hp756

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

ProbeError

Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Unhealthy

Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

ProbeError

Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Unhealthy

Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-inf-b-31323b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-31323b-openshift-default-868b86bb4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-b-31323b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-31323b

openshift-ingress

replicaset-controller

e2e-inf-b-31323b-openshift-default-868b86bb4

SuccessfulCreate

Created pod: e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

openshift-ingress

multus

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-31323b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-31323b-54f9d64c48 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-31323b: serviceaccount "payload-processing-e2e-inf-b-31323b" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-31323b-754cdb9876

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-31323b-754cdb9876-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-31323b: serviceaccount "payload-processing-e2e-inf-b-31323b" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-31323b-754cdb9876

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-31323b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-31323b-754cdb9876 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

Created

Created container: payload-processing

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Unhealthy

Readiness probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

ProbeError

Readiness probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

ProbeError

Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Unhealthy

Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

Unhealthy

Readiness probe failed: dial tcp 10.132.0.69:9004: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-inf-a-55901c-596775948d-hp756

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-55901c-596775948d-hp756

Unhealthy

Readiness probe failed: dial tcp 10.132.0.70:9004: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-55901c-86895d48c6-lmzjz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-a-55901c-openshift-default-5466b565dd-gxbl6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-inf-b-31323b-754cdb9876-w9rkl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-31323b-54f9d64c48-9xgtq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-api-a-62afff-openshift-default-7cb7dbd894

SuccessfulCreate

Created pod: e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

openshift-ingress

deployment-controller

e2e-api-a-62afff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-62afff-openshift-default-7cb7dbd894 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-a-62afff

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-62afff

openshift-ingress

multus

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-b-31323b-openshift-default-868b86bb4-pfjtk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-62afff-555987f44b

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-62afff-555987f44b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-62afff: serviceaccount "payload-processing-e2e-api-a-62afff" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-62afff

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-62afff-555987f44b from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-62afff

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-62afff-65bbffbd4d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-62afff: serviceaccount "payload-processing-e2e-api-a-62afff" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-62afff-555987f44b

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-62afff-555987f44b-fhchj

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-62afff-555987f44b-fhchj

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-62afff-555987f44b-fhchj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-62afff-555987f44b-fhchj

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-api-a-62afff-555987f44b-fhchj

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

ProbeError

Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": read tcp 10.134.0.2:34598->10.134.0.58:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Unhealthy

Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": read tcp 10.134.0.2:34598->10.134.0.58:15021: read: connection reset by peer
(x5)

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-api-b-9e17b5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-9e17b5

openshift-ingress

multus

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-b-9e17b5-openshift-default-744c4bd55b

SuccessfulCreate

Created pod: e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

openshift-ingress

deployment-controller

e2e-api-b-9e17b5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-9e17b5-openshift-default-744c4bd55b from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-9e17b5-5d77649fd8

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-9e17b5-5d77649fd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9e17b5: serviceaccount "payload-processing-e2e-api-b-9e17b5" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-9e17b5: serviceaccount "payload-processing-e2e-api-b-9e17b5" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-9e17b5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-9e17b5-5d77649fd8 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-9e17b5-5d77649fd8

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-9e17b5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

ProbeError

Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Unhealthy

Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-62afff-65bbffbd4d-rw5t6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-62afff-555987f44b-fhchj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-a-62afff-openshift-default-7cb7dbd894-w7l4w

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-9e17b5-585bdd56f6-pjmsn

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-9e17b5-5d77649fd8-ht72x

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-a-a7013e-openshift-default-7886c8454d

SuccessfulCreate

Created pod: e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

openshift-ingress

gateway_labeler_controller

e2e-shared-a-a7013e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-a7013e

openshift-ingress

multus

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-a-a7013e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-a7013e-openshift-default-7886c8454d from 0 to 1

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-b-9e17b5-openshift-default-744c4bd55b-sn77n

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-a7013e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-a7013e-79b4888dd6

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-a7013e-79b4888dd6

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-a7013e-79b4888dd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-a7013e: serviceaccount "payload-processing-e2e-shared-a-a7013e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-a7013e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-a7013e-79b4888dd6 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-a7013e: serviceaccount "payload-processing-e2e-shared-a-a7013e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

openshift-ingress

kubelet

payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Unhealthy

Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": read tcp 10.134.0.2:52408->10.134.0.60:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

ProbeError

Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": read tcp 10.134.0.2:52408->10.134.0.60:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Unhealthy

Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

ProbeError

Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-shared-b-038ca6-openshift-default-684685fc4d

SuccessfulCreate

Created pod: e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

openshift-ingress

gateway_labeler_controller

e2e-shared-b-038ca6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-038ca6

openshift-ingress

deployment-controller

e2e-shared-b-038ca6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-038ca6-openshift-default-684685fc4d from 0 to 1

openshift-ingress

multus

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-038ca6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-038ca6-75f7cc9cf4 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-038ca6: serviceaccount "payload-processing-e2e-shared-b-038ca6" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-038ca6: serviceaccount "payload-processing-e2e-shared-b-038ca6" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-038ca6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

ProbeError

Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Unhealthy

Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Unhealthy

Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

ProbeError

Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-a888a4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a888a4

openshift-ingress

replicaset-controller

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6

SuccessfulCreate

Created pod: e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

openshift-ingress

multus

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ipp-a-a888a4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a888a4: serviceaccount "payload-processing-e2e-ipp-a-a888a4" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-a888a4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-a888a4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-a888a4-5b5f6b556b from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a888a4: serviceaccount "payload-processing-e2e-ipp-a-a888a4" not found

openshift-ingress

multus

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

ProbeError

Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Unhealthy

Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Unhealthy

Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

ProbeError

Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-ipp-b-0778c1-openshift-default-c877584d4

SuccessfulCreate

Created pod: e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-0778c1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-0778c1

openshift-ingress

deployment-controller

e2e-ipp-b-0778c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-0778c1-openshift-default-c877584d4 from 0 to 1

openshift-ingress

kubelet

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

multus

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0778c1: serviceaccount "payload-processing-e2e-ipp-b-0778c1" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-0778c1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4 from 0 to 1
(x2)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0778c1: serviceaccount "payload-processing-e2e-ipp-b-0778c1" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-0778c1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-0778c1-cd55fbdb8 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

Created

Created container: payload-pre-processing

openshift-ingress

GatewayAuthBootstrap

e2e-ipp-cleanup-672896

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ipp-cleanup-672896-authn-ssl: envoyfilters.networking.istio.io "e2e-ipp-cleanup-672896-authn-ssl" already exists

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb

SuccessfulCreate

Created pod: e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

openshift-ingress

multus

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-672896

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-672896

openshift-ingress

deployment-controller

e2e-ipp-cleanup-672896-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-672896

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-672896: serviceaccount "payload-processing-e2e-ipp-cleanup-672896" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-672896

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-672896-768fdd448 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-672896-768fdd448

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-672896-768fdd448

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-672896-768fdd448-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-672896: serviceaccount "payload-processing-e2e-ipp-cleanup-672896" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

ProbeError

Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Unhealthy

Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

ProbeError

Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Unhealthy

Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-672896-768fdd448-rn756

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-672896-75776848b8-hg6g6

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-cleanup-672896

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-cleanup-672896": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-672896, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a6c305f8-f111-4746-a340-27757a09796d, UID in object meta:

openshift-ingress

kubelet

e2e-ipp-cleanup-672896-openshift-default-79db7b8cfb-lj489

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a888a4-5b5f6b556b-q745d

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-a-a888a4

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-a888a4": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-a888a4, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: dfa21ae8-96c4-4af4-ae0e-0a83797930a5, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a888a4-6cc4944696-zdv8h

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-a888a4-openshift-default-5bcb99d7d6-hp2m5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0778c1-cd55fbdb8-d7rjf

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-b-0778c1

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-0778c1": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-0778c1, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 75949eb7-7373-4ad6-9ef6-387c28793983, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0778c1-59c6b896b4-hjprb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

ProbeError

Readiness probe error: Get "http://10.134.0.69:15021/healthz/ready": read tcp 10.134.0.2:34494->10.134.0.69:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-ipp-b-0778c1-openshift-default-c877584d4-smx8f

Unhealthy

Readiness probe failed: Get "http://10.134.0.69:15021/healthz/ready": read tcp 10.134.0.2:34494->10.134.0.69:15021: read: connection reset by peer

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-a7013e-bbbd45cb4-h46vs

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-a7013e-79b4888dd6-5ljbv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-a7013e-openshift-default-7886c8454d-5d7gh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-038ca6-75f7cc9cf4-9rc49

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-038ca6-65cc67cb4-h6kft

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-038ca6

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-038ca6": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-038ca6, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 5fe4d267-9f3a-48d1-bca0-65e30186e0af, UID in object meta:

openshift-ingress

kubelet

e2e-shared-b-038ca6-openshift-default-684685fc4d-2pxn5

Killing

Stopping container istio-proxy