Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-654463-openshift-default-8bf48d448-d6262 to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b to ip-10-0-128-30.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz to ip-10-0-143-147.ec2.internal

openshift-ingress

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-mt-93683709-openshift-default-6df68648-ckchx

Scheduled

Successfully assigned openshift-ingress/e2e-mt-93683709-openshift-default-6df68648-ckchx to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

Scheduled

Successfully assigned openshift-ingress/e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd to ip-10-0-128-30.ec2.internal

openshift-ingress

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb to ip-10-0-128-30.ec2.internal

openshift-ingress

payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9 to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt to ip-10-0-143-147.ec2.internal

openshift-ingress

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72 to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Scheduled

Successfully assigned openshift-ingress/e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26 to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9 to ip-10-0-142-104.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

Scheduled

Successfully assigned openshift-ingress/e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9 to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw to ip-10-0-142-104.ec2.internal

openshift-ingress

kube-auth-proxy-5bbd68dc59-wktc5

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5bbd68dc59-wktc5 to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg to ip-10-0-143-147.ec2.internal

openshift-ingress

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

Scheduled

Successfully assigned openshift-ingress/e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b to ip-10-0-128-30.ec2.internal

openshift-ingress

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6 to ip-10-0-142-104.ec2.internal

openshift-ingress

kube-auth-proxy-5bbd68dc59-6grwd

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-5bbd68dc59-6grwd to ip-10-0-128-30.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-rzl5d

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-rzl5d to ip-10-0-142-104.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-f2pj4 to ip-10-0-128-30.ec2.internal

openshift-ingress

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv to ip-10-0-142-104.ec2.internal

openshift-ingress

router-default-75b6d7b44b-fvj8r

Scheduled

Successfully assigned openshift-ingress/router-default-75b6d7b44b-fvj8r to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8 to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7 to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s to ip-10-0-128-30.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg to ip-10-0-143-147.ec2.internal

openshift-ingress

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn to ip-10-0-128-30.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-jbzm5

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-jbzm5 to ip-10-0-142-104.ec2.internal

openshift-ingress

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

Scheduled

Successfully assigned openshift-ingress/e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts to ip-10-0-128-30.ec2.internal

openshift-ingress

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf to ip-10-0-143-147.ec2.internal

openshift-ingress

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts to ip-10-0-128-30.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-4s9v7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-4s9v7 to ip-10-0-143-147.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-v57nj

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-v57nj to ip-10-0-128-30.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc to ip-10-0-143-147.ec2.internal

openshift-ingress

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Scheduled

Successfully assigned openshift-ingress/e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw to ip-10-0-142-104.ec2.internal

openshift-ingress

payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-d887938f-644585b75c-h5xkv to ip-10-0-143-147.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2 to ip-10-0-142-104.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-75b6d7b44b from 0 to 1
(x10)

openshift-ingress

replicaset-controller

router-default-75b6d7b44b

FailedCreate

Error creating: pods "router-default-75b6d7b44b-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x17)

openshift-ingress

replicaset-controller

router-default-75b6d7b44b

FailedCreate

Error creating: pods "router-default-75b6d7b44b-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-75b6d7b44b

SuccessfulCreate

Created pod: router-default-75b6d7b44b-fvj8r
(x5)

openshift-ingress

kubelet

router-default-75b6d7b44b-fvj8r

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-75b6d7b44b-fvj8r

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-75b6d7b44b-fvj8r

Started

Started container router

openshift-ingress

kubelet

router-default-75b6d7b44b-fvj8r

Created

Created container: router

openshift-ingress

multus

router-default-75b6d7b44b-fvj8r

AddedInterface

Add eth0 [10.134.0.12/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-75b6d7b44b-fvj8r

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eaa467bc105be2f7695ecfea0f0c53f4c5cd52f1f705da378c7a33d13f1d102f" already present on machine
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-5bbd68dc59

FailedCreate

Error creating: pods "kube-auth-proxy-5bbd68dc59-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-5bbd68dc59 from 0 to 2

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-6grwd

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

kube-auth-proxy-5bbd68dc59

SuccessfulCreate

Created pod: kube-auth-proxy-5bbd68dc59-wktc5

openshift-ingress

replicaset-controller

kube-auth-proxy-5bbd68dc59

SuccessfulCreate

Created pod: kube-auth-proxy-5bbd68dc59-6grwd

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-wktc5

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-5bbd68dc59-wktc5

AddedInterface

Add eth0 [10.132.0.22/23] from ovn-kubernetes

openshift-ingress

multus

kube-auth-proxy-5bbd68dc59-6grwd

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-v57nj

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-v57nj

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-v57nj

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-wktc5

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-6grwd

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-6grwd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.979s (2.979s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-6grwd

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-wktc5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.84s (2.84s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-5bbd68dc59-wktc5

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-v57nj

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-v57nj

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-v57nj

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.518s (3.518s including waiting). Image size: 181126104 bytes.

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-6bslg

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.802s (2.802s including waiting). Image size: 181318953 bytes.

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-rzl5d

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-rzl5d

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-rzl5d

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-rzl5d

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.273s (2.273s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-rzl5d

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-rzl5d

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-rzl5d

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-rzl5d

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-v57nj

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-v57nj

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-f2pj4

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.534s (2.534s including waiting). Image size: 181318953 bytes.
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-4s9v7

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-jbzm5

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-4s9v7

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-4s9v7

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

multus

payload-processing-5b54b7fbb7-jbzm5

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-jbzm5

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-jbzm5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.29s (1.29s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-jbzm5

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-jbzm5

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-4s9v7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-4s9v7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-4s9v7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.847s (2.847s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Unhealthy

Readiness probe failed: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

ProbeError

Readiness probe error: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

Unhealthy

Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-f2pj4

ProbeError

Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-worker-w4-654463-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-654463-openshift-default-8bf48d448 from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-a-7c4f4b-openshift-default-6d485db978

SuccessfulCreate

Created pod: e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

openshift-ingress

deployment-controller

e2e-worker-w0-946609-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-946609-openshift-default-7d4bdddd56 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-946609

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-946609

openshift-ingress

multus

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-654463-openshift-default-8bf48d448

SuccessfulCreate

Created pod: e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

openshift-ingress

deployment-controller

e2e-inf-a-7c4f4b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-7c4f4b-openshift-default-6d485db978 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-654463

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-654463

openshift-ingress

gateway_labeler_controller

e2e-inf-a-7c4f4b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-7c4f4b

openshift-ingress

multus

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-946609-openshift-default-7d4bdddd56

SuccessfulCreate

Created pod: e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

openshift-ingress

multus

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-654463-db8fb8547

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-654463-db8fb8547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-654463: serviceaccount "payload-processing-e2e-worker-w4-654463" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-654463-64f88b955d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-654463-64f88b955d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-654463-64f88b955d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-654463: serviceaccount "payload-processing-e2e-worker-w4-654463" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-654463

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-654463-64f88b955d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-654463-db8fb8547

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-654463

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-654463-db8fb8547 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-7c4f4b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-7c4f4b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-7c4f4b: serviceaccount "payload-processing-e2e-inf-a-7c4f4b" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-7c4f4b: serviceaccount "payload-processing-e2e-inf-a-7c4f4b" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-946609-9fd88897f

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

openshift-ingress

multus

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-946609

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-946609-67f4478978 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-946609-9fd88897f

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-946609-9fd88897f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-946609: serviceaccount "payload-processing-e2e-worker-w0-946609" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-946609

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-946609-9fd88897f from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-946609-67f4478978

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-946609-67f4478978-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-946609: serviceaccount "payload-processing-e2e-worker-w0-946609" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-946609-67f4478978

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

Started

Started container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-autores-43fdfe

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-43fdfe

openshift-ingress

deployment-controller

e2e-autores-43fdfe-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-43fdfe-openshift-default-7bc9f8d47d from 0 to 1

openshift-ingress

replicaset-controller

e2e-autores-43fdfe-openshift-default-7bc9f8d47d

SuccessfulCreate

Created pod: e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

Created

Created container: payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-0da33e9c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-0da33e9c

openshift-ingress

replicaset-controller

e2e-ait-0da33e9c-openshift-default-fbf476fc6

SuccessfulCreate

Created pod: e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

openshift-ingress

deployment-controller

e2e-ait-0da33e9c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-0da33e9c-openshift-default-fbf476fc6 from 0 to 1

openshift-ingress

multus

e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

multus

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

Created

Created container: istio-proxy
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-0da33e9c-68795bc8cf

FailedCreate

Error creating: pods "payload-processing-e2e-ait-0da33e9c-68795bc8cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0da33e9c: serviceaccount "payload-processing-e2e-ait-0da33e9c" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-0da33e9c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-0da33e9c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-0da33e9c-68795bc8cf from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0da33e9c: serviceaccount "payload-processing-e2e-ait-0da33e9c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

openshift-ingress

kubelet

payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-0da33e9c-68795bc8cf

SuccessfulCreate

Created pod: payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

openshift-ingress

multus

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-43fdfe-6bbcb85f7

SuccessfulCreate

Created pod: payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-43fdfe

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-43fdfe-7f95cfd466 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-43fdfe: serviceaccount "payload-processing-e2e-autores-43fdfe" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-43fdfe-6bbcb85f7

FailedCreate

Error creating: pods "payload-processing-e2e-autores-43fdfe-6bbcb85f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-43fdfe: serviceaccount "payload-processing-e2e-autores-43fdfe" not found

openshift-ingress

deployment-controller

payload-processing-e2e-autores-43fdfe

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-43fdfe-6bbcb85f7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

e2e-inf-b-708b6f-openshift-default-79f59fdd86

SuccessfulCreate

Created pod: e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

openshift-ingress

deployment-controller

e2e-inf-b-708b6f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-708b6f-openshift-default-79f59fdd86 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-b-708b6f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-708b6f
(x2)

openshift-ingress

kubelet

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

Created

Created container: istio-proxy

openshift-ingress

multus

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-708b6f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-708b6f-74c477c5d9

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-708b6f-74c477c5d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-708b6f: serviceaccount "payload-processing-e2e-inf-b-708b6f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-708b6f: serviceaccount "payload-processing-e2e-inf-b-708b6f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-708b6f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-708b6f-74c477c5d9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-708b6f-74c477c5d9

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

openshift-ingress

kubelet

payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-0da33e9c

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-0da33e9c: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-0da33e9c": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-0da33e9c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: fe3361f9-907a-41af-9be4-3981ccc7841d, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0da33e9c-6974b99bfd-cvqm8

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ait-0da33e9c-68795bc8cf-qqsg9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-0da33e9c-openshift-default-fbf476fc6-j8nbd

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d

SuccessfulCreate

Created pod: e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

openshift-ingress

gateway_labeler_controller

e2e-migrate-994a4fd1-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-994a4fd1-gw

openshift-ingress

deployment-controller

e2e-migrate-994a4fd1-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-994a4fd1-gw-openshift-default-d5785574d from 0 to 1

openshift-ingress

multus

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

ProbeError

Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-994a4fd1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-994a4fd1-795478bffd from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-994a4fd1-795478bffd

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-994a4fd1-795478bffd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-994a4fd1: serviceaccount "payload-processing-e2e-migrate-994a4fd1" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-994a4fd1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-994a4fd1-679b5c994 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-994a4fd1: serviceaccount "payload-processing-e2e-migrate-994a4fd1" not found
(x6)

openshift-ingress

kubelet

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

Unhealthy

Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-994a4fd1-795478bffd

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

openshift-ingress

kubelet

payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Created

Created container: istio-proxy
(x6)

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

ProbeError

Startup probe error: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Unhealthy

Startup probe failed: Get "http://10.133.0.50:15021/healthz/ready": dial tcp 10.133.0.50:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-migrate-994a4fd1-795478bffd-whvsf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-994a4fd1-679b5c994-xpcgk

Killing

Stopping container payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-5e4c6960

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-5e4c6960

openshift-ingress

deployment-controller

e2e-ait-5e4c6960-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-5e4c6960-openshift-default-66f49f6f9b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-5e4c6960-openshift-default-66f49f6f9b

SuccessfulCreate

Created pod: e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

openshift-ingress

multus

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-5e4c6960-7dd6957f4c

SuccessfulCreate

Created pod: payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-5e4c6960-7dd6957f4c

FailedCreate

Error creating: pods "payload-processing-e2e-ait-5e4c6960-7dd6957f4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5e4c6960: serviceaccount "payload-processing-e2e-ait-5e4c6960" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-5e4c6960

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-5e4c6960

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-5e4c6960-7dd6957f4c from 0 to 1

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

openshift-ingress

kubelet

e2e-migrate-994a4fd1-gw-openshift-default-d5785574d-6p4bg

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-5e4c6960: serviceaccount "payload-processing-e2e-ait-5e4c6960" not found

openshift-ingress

multus

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Unhealthy

Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

ProbeError

Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ait-5e4c6960-7dd6957f4c-2sjlz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-5e4c6960-65b7c74fdb-s7485

Killing

Stopping container payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-derive-246af4f6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-246af4f6

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-246af4f6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-246af4f6-openshift-default-6876dc689b from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-246af4f6-openshift-default-6876dc689b

SuccessfulCreate

Created pod: e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

openshift-ingress

GatewayAuthBootstrap

e2e-derive-246af4f6

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-derive-246af4f6-authn-ssl: envoyfilters.networking.istio.io "e2e-derive-246af4f6-authn-ssl" already exists

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-5e4c6960-openshift-default-66f49f6f9b-hrncc

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-246af4f6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-246af4f6-ccf5476c8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-derive-246af4f6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-246af4f6-64b9cd8c8f from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-246af4f6-64b9cd8c8f

SuccessfulCreate

Created pod: payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-246af4f6: serviceaccount "payload-processing-e2e-derive-246af4f6" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-246af4f6-64b9cd8c8f

FailedCreate

Error creating: pods "payload-processing-e2e-derive-246af4f6-64b9cd8c8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-246af4f6: serviceaccount "payload-processing-e2e-derive-246af4f6" not found

openshift-ingress

kubelet

payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

ProbeError

Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Unhealthy

Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-246af4f6-ccf5476c8-974q4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-derive-246af4f6-64b9cd8c8f-grqfh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-derive-246af4f6-openshift-default-6876dc689b-gkppw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-43fdfe-7f95cfd466-fdq2j

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

Unhealthy

Readiness probe failed: dial tcp 10.134.0.50:9004: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-autores-43fdfe-6bbcb85f7-wv7k8

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-autores-43fdfe

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-autores-43fdfe: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-autores-43fdfe": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-autores-43fdfe, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 2134ef3d-886d-4e18-8fa9-4974a093cc18, UID in object meta:

openshift-ingress

kubelet

e2e-autores-43fdfe-openshift-default-7bc9f8d47d-9ntts

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-explicit-14ee59-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-14ee59-openshift-default-77dc4b89fb from 0 to 1

openshift-ingress

replicaset-controller

e2e-explicit-14ee59-openshift-default-77dc4b89fb

SuccessfulCreate

Created pod: e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

openshift-ingress

GatewayAuthBootstrap

e2e-explicit-14ee59

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-explicit-14ee59-authn-ssl: envoyfilters.networking.istio.io "e2e-explicit-14ee59-authn-ssl" already exists

openshift-ingress

gateway_labeler_controller

e2e-explicit-14ee59

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-14ee59

openshift-ingress

multus

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-14ee59

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-14ee59-7784b6b66f from 0 to 1

openshift-ingress

multus

payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-14ee59

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-14ee59-7b9c6754db from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-14ee59-7784b6b66f

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-14ee59-7784b6b66f

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-14ee59-7784b6b66f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-14ee59: serviceaccount "payload-processing-e2e-explicit-14ee59" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-14ee59: serviceaccount "payload-processing-e2e-explicit-14ee59" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

ProbeError

Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Unhealthy

Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-mt-d887938f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-d887938f-openshift-default-7d5ddb489b from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-d887938f-openshift-default-7d5ddb489b

SuccessfulCreate

Created pod: e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

openshift-ingress

gateway_labeler_controller

e2e-mt-d887938f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d887938f

openshift-ingress

multus

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-d887938f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-d887938f-5645b797b6 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d887938f-644585b75c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-d887938f-644585b75c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d887938f: serviceaccount "payload-processing-e2e-mt-d887938f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-d887938f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-d887938f-644585b75c from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d887938f-5645b797b6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-d887938f-5645b797b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d887938f: serviceaccount "payload-processing-e2e-mt-d887938f" not found

openshift-ingress

multus

payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d887938f-5645b797b6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

openshift-ingress

kubelet

payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d887938f-644585b75c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

ProbeError

Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

Unhealthy

Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-d887938f

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-d887938f: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-d887938f": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-d887938f, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: bbc89f5e-bc26-4b7c-9290-98764c582f80, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-mt-d887938f-644585b75c-h5xkv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d887938f-5645b797b6-q8vkj

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-d887938f-openshift-default-7d5ddb489b-bw7kr

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-bac224

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-bac224

openshift-ingress

replicaset-controller

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb

SuccessfulCreate

Created pod: e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

openshift-ingress

deployment-controller

e2e-shared-a-w2-bac224-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb from 0 to 1

openshift-ingress

multus

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-bac224

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-bac224-58676c465c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-bac224-58676c465c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-bac224: serviceaccount "payload-processing-e2e-shared-a-w2-bac224" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-bac224-58676c465c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-bac224

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-bac224-58676c465c from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-bac224: serviceaccount "payload-processing-e2e-shared-a-w2-bac224" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Unhealthy

Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": read tcp 10.134.0.2:41814->10.134.0.74:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

ProbeError

Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": read tcp 10.134.0.2:41814->10.134.0.74:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8

SuccessfulCreate

Created pod: e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

openshift-ingress

deployment-controller

e2e-shared-b-w2-87d618-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8 from 0 to 1

openshift-ingress

multus

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-87d618

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-87d618
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Unhealthy

Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

ProbeError

Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-explicit-14ee59-7784b6b66f-2bcg9

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-explicit-14ee59

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-explicit-14ee59: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-explicit-14ee59": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-explicit-14ee59, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ae1abb7d-f4b6-46ad-b5be-502a63277384, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-14ee59-7b9c6754db-58xcm

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-87d618: serviceaccount "payload-processing-e2e-shared-b-w2-87d618" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-87d618

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-87d618

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-87d618: serviceaccount "payload-processing-e2e-shared-b-w2-87d618" not found

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c

Started

Started container payload-processing

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-explicit-14ee59-openshift-default-77dc4b89fb-gkdr6

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-noten-2f5ffe

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-2f5ffe

openshift-ingress

kubelet

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

multus

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-noten-2f5ffe-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-2f5ffe-openshift-default-5c68bc56b4 from 0 to 1

openshift-ingress

replicaset-controller

e2e-noten-2f5ffe-openshift-default-5c68bc56b4

SuccessfulCreate

Created pod: e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-mt-93683709-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-93683709-openshift-default-6df68648 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-93683709-openshift-default-6df68648

SuccessfulCreate

Created pod: e2e-mt-93683709-openshift-default-6df68648-ckchx

openshift-ingress

gateway_labeler_controller

e2e-mt-93683709

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-93683709
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:

openshift-ingress

multus

e2e-mt-93683709-openshift-default-6df68648-ckchx

AddedInterface

Add eth0 [10.134.0.81/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-93683709-564fcc4fb9

FailedCreate

Error creating: pods "payload-processing-e2e-mt-93683709-564fcc4fb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-93683709: serviceaccount "payload-processing-e2e-mt-93683709" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-93683709

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-93683709-7c56d87bc8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-93683709-7c56d87bc8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-93683709-564fcc4fb9

SuccessfulCreate

Created pod: payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-93683709-7c56d87bc8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-93683709-7c56d87bc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-93683709: serviceaccount "payload-processing-e2e-mt-93683709" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-93683709

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-93683709-564fcc4fb9 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-7c4f4b-676bf77998-zd5nt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-7c4f4b-f9bdcbd65-w2rg9

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-7c4f4b

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-7c4f4b: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-7c4f4b": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-7c4f4b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 1047cab2-1ee6-4cd1-b623-516bd33d0a26, UID in object meta:

openshift-ingress

kubelet

e2e-inf-a-7c4f4b-openshift-default-6d485db978-sn4vw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-708b6f-654cf9f9db-mfx72

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-708b6f-74c477c5d9-hk8ph

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-mt-93683709-openshift-default-6df68648-ckchx

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-93683709-openshift-default-6df68648-ckchx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-93683709-openshift-default-6df68648-ckchx

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-93683709-openshift-default-6df68648-ckchx

Unhealthy

Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-93683709-openshift-default-6df68648-ckchx

ProbeError

Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd

SuccessfulCreate

Created pod: e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

openshift-ingress

deployment-controller

e2e-shared-a-w3-ec2077-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-ec2077

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-ec2077

openshift-ingress

multus

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-b-708b6f-openshift-default-79f59fdd86-lfg5b

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-2f5ffe-openshift-default-5c68bc56b4-65gp9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ec2077: serviceaccount "payload-processing-e2e-shared-a-w3-ec2077" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-ec2077

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-ec2077

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-ec2077: serviceaccount "payload-processing-e2e-shared-a-w3-ec2077" not found

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Unhealthy

Readiness probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

deployment-controller

e2e-shared-b-w3-8fdd5b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb

SuccessfulCreate

Created pod: e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-8fdd5b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-8fdd5b

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

ProbeError

Readiness probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Created

Created container: istio-proxy

openshift-ingress

multus

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Started

Started container istio-proxy
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8fdd5b: serviceaccount "payload-processing-e2e-shared-b-w3-8fdd5b" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8fdd5b: serviceaccount "payload-processing-e2e-shared-b-w3-8fdd5b" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-8fdd5b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-8fdd5b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

AddedInterface

Add eth0 [10.134.0.84/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Unhealthy

Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

AddedInterface

Add eth0 [10.134.0.85/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

ProbeError

Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

Unhealthy

Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

ProbeError

Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-93683709-7c56d87bc8-l5mcg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-93683709-564fcc4fb9-zprdk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-93683709-openshift-default-6df68648-ckchx

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-api-a-2686de

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-2686de

openshift-ingress

replicaset-controller

e2e-api-a-2686de-openshift-default-78c78f86c8

SuccessfulCreate

Created pod: e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

openshift-ingress

deployment-controller

e2e-api-a-2686de-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-2686de-openshift-default-78c78f86c8 from 0 to 1

openshift-ingress

multus

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

AddedInterface

Add eth0 [10.134.0.87/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-2686de-549bdbcdb9

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-2686de-549bdbcdb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2686de: serviceaccount "payload-processing-e2e-api-a-2686de" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-2686de-667c569ffd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-2686de-667c569ffd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-2686de: serviceaccount "payload-processing-e2e-api-a-2686de" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-2686de

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-2686de-667c569ffd from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-2686de-549bdbcdb9

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-2686de

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-2686de-549bdbcdb9 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-2686de-667c569ffd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

openshift-ingress

kubelet

payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

AddedInterface

Add eth0 [10.134.0.88/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

e2e-api-b-bcb767-openshift-default-7d6844fdb4

SuccessfulCreate

Created pod: e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

openshift-ingress

gateway_labeler_controller

e2e-api-b-bcb767

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-bcb767

openshift-ingress

deployment-controller

e2e-api-b-bcb767-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-bcb767-openshift-default-7d6844fdb4 from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

ProbeError

Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused body:

openshift-ingress

multus

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

AddedInterface

Add eth0 [10.134.0.89/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Unhealthy

Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-bcb767-86b5b575f9

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-bcb767-86b5b575f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bcb767: serviceaccount "payload-processing-e2e-api-b-bcb767" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bcb767: serviceaccount "payload-processing-e2e-api-b-bcb767" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-bcb767

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-bcb767-86b5b575f9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-bcb767-86b5b575f9

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-bcb767

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-bcb767-ddbf7877d from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

AddedInterface

Add eth0 [10.134.0.90/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

AddedInterface

Add eth0 [10.134.0.91/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Unhealthy

Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

ProbeError

Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-2686de-549bdbcdb9-bzgvc

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-a-2686de

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-a-2686de: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-a-2686de": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-a-2686de, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 524b852a-11ea-4e3f-9823-2a24a3024d17, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-2686de-667c569ffd-brnhm

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

ProbeError

Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Unhealthy

Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-a-2686de-openshift-default-78c78f86c8-2g6h9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-bcb767-ddbf7877d-dvwlf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-bcb767-86b5b575f9-lltb5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-946609-9fd88897f-68ljl

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-worker-w0-946609

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-worker-w0-946609: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-worker-w0-946609": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-worker-w0-946609, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 7ac44903-1075-44f4-bc93-09e99b1b6579, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-946609-67f4478978-fmbp7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-b-bcb767-openshift-default-7d6844fdb4-qv48d

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-946609-openshift-default-7d4bdddd56-p9h5b

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-bac224-6b5977dbc4-wrfrp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-bac224-58676c465c-2j9m2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w2-bac224-openshift-default-7f8896f4bb-bcsqp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-654463-64f88b955d-dctsg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-654463-db8fb8547-x6klj

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-worker-w4-654463

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-worker-w4-654463: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-worker-w4-654463": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-worker-w4-654463, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 06abc140-439d-4d58-85e5-bcd29bd6dd87, UID in object meta:

openshift-ingress

kubelet

e2e-worker-w4-654463-openshift-default-8bf48d448-d6262

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-87d618-6c8d8db6bb-m75bv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-87d618-5d94f66f4c-kwt2c

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w2-87d618

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w2-87d618: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w2-87d618": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w2-87d618, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 5936ed03-5ab1-4bf7-9896-4cdfedf64a39, UID in object meta:

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w2-87d618-openshift-default-86d5f9b9b8-r4nfn

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

e2e-ipp-a-a7bd58-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-a7bd58-openshift-default-5658f87fb8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8

SuccessfulCreate

Created pod: e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-a7bd58

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a7bd58

openshift-ingress

multus

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

AddedInterface

Add eth0 [10.134.0.97/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a7bd58: serviceaccount "payload-processing-e2e-ipp-a-a7bd58" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-a7bd58-6999dbc775

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-a7bd58

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-a7bd58-6999dbc775 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-a7bd58-6999dbc775

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-a7bd58-6999dbc775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a7bd58: serviceaccount "payload-processing-e2e-ipp-a-a7bd58" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-a7bd58

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Unhealthy

Readiness probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

ProbeError

Readiness probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Unhealthy

Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

ProbeError

Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-ipp-b-1672e6-openshift-default-7465df8846

SuccessfulCreate

Created pod: e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

openshift-ingress

deployment-controller

e2e-ipp-b-1672e6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-1672e6-openshift-default-7465df8846 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-1672e6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-1672e6

openshift-ingress

multus

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

AddedInterface

Add eth0 [10.133.0.82/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-1672e6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-1672e6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-1672e6-58bfdf58c7 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1672e6: serviceaccount "payload-processing-e2e-ipp-b-1672e6" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1672e6: serviceaccount "payload-processing-e2e-ipp-b-1672e6" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

ProbeError

Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

Unhealthy

Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4

SuccessfulCreate

Created pod: e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

openshift-ingress

deployment-controller

e2e-ipp-cleanup-75ce90-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-75ce90

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-75ce90

openshift-ingress

multus

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

AddedInterface

Add eth0 [10.134.0.105/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-75ce90

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-75ce90: serviceaccount "payload-processing-e2e-ipp-cleanup-75ce90" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-75ce90: serviceaccount "payload-processing-e2e-ipp-cleanup-75ce90" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-75ce90

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

AddedInterface

Add eth0 [10.134.0.106/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

ProbeError

Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

Unhealthy

Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-75ce90-86f8c79549-29njf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-75ce90-58595b76f8-ljc26

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-75ce90-openshift-default-854df5bff4-nrspp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-a7bd58-6999dbc775-qcqlf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-a7bd58-54468db6f8-4tdxf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-a7bd58-openshift-default-5658f87fb8-t8chk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-1672e6-58bfdf58c7-7mvlb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-1672e6-b445c6b9d-lsp7s

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-1672e6-openshift-default-7465df8846-4slts

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-ec2077-977d8c666-h6jvk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-ec2077-99f46f5c9-d9jtc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-a-w3-ec2077-openshift-default-db94dcdbd-zbvvb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-8fdd5b-f766cfdc5-j9jrv

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w3-8fdd5b

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8fdd5b: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w3-8fdd5b": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8fdd5b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 298c31e0-b7f2-484a-af7a-a9cfb9bf2053, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-8fdd5b-76c7c7b74-z97ww

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w3-8fdd5b-openshift-default-d4c5bb5fb-zm45s

Killing

Stopping container istio-proxy