Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478 to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96 to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6 to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8 to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

Scheduled

Successfully assigned openshift-ingress/e2e-ait-18c04da2-openshift-default-555c79855d-vth4r to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9 to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-1910e9-openshift-default-68f655766-s56th to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4 to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6 to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Scheduled

Successfully assigned openshift-ingress/e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6 to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884 to ip-10-0-133-125.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-b2j7q

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b2j7q to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d to ip-10-0-135-139.ec2.internal

openshift-ingress

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss to ip-10-0-135-139.ec2.internal

openshift-ingress

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-d220da-openshift-default-74b6789978-29pww to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Scheduled

Successfully assigned openshift-ingress/e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6 to ip-10-0-141-156.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-vkhlm

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-vkhlm to ip-10-0-135-139.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7 to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8 to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z to ip-10-0-135-139.ec2.internal

openshift-ingress

kube-auth-proxy-cb779b95-ww4qf

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-cb779b95-ww4qf to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8 to ip-10-0-133-125.ec2.internal

openshift-ingress

router-default-6bbc499686-jdqd8

Scheduled

Successfully assigned openshift-ingress/router-default-6bbc499686-jdqd8 to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

Scheduled

Successfully assigned openshift-ingress/e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt to ip-10-0-135-139.ec2.internal

openshift-ingress

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57 to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5 to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-wkmjz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-wkmjz to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v to ip-10-0-135-139.ec2.internal

openshift-ingress

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Scheduled

Successfully assigned openshift-ingress/e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt to ip-10-0-135-139.ec2.internal

openshift-ingress

e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

Scheduled

Successfully assigned openshift-ingress/e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8 to ip-10-0-135-139.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-ncl9j

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-ncl9j to ip-10-0-141-156.ec2.internal

openshift-ingress

kube-auth-proxy-cb779b95-vtsgd

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-cb779b95-vtsgd to ip-10-0-141-156.ec2.internal

openshift-ingress

payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq to ip-10-0-135-139.ec2.internal

openshift-ingress

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8 to ip-10-0-135-139.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-7kkbk to ip-10-0-141-156.ec2.internal

openshift-ingress

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Scheduled

Successfully assigned openshift-ingress/e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp to ip-10-0-133-125.ec2.internal

openshift-ingress

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg to ip-10-0-133-125.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw to ip-10-0-135-139.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-6bbc499686 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x13)

openshift-ingress

replicaset-controller

router-default-6bbc499686

FailedCreate

Error creating: pods "router-default-6bbc499686-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-6bbc499686

FailedCreate

Error creating: pods "router-default-6bbc499686-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

replicaset-controller

router-default-6bbc499686

SuccessfulCreate

Created pod: router-default-6bbc499686-jdqd8
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x6)

openshift-ingress

kubelet

router-default-6bbc499686-jdqd8

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-6bbc499686-jdqd8

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-6bbc499686-jdqd8

AddedInterface

Add eth0 [10.132.0.14/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-6bbc499686-jdqd8

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

kubelet

router-default-6bbc499686-jdqd8

Created

Created container: router

openshift-ingress

kubelet

router-default-6bbc499686-jdqd8

Started

Started container router

openshift-ingress

replicaset-controller

kube-auth-proxy-cb779b95

SuccessfulCreate

Created pod: kube-auth-proxy-cb779b95-vtsgd

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-ww4qf

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-cb779b95 from 0 to 2

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-vtsgd

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-cb779b95

SuccessfulCreate

Created pod: kube-auth-proxy-cb779b95-ww4qf
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-cb779b95

FailedCreate

Error creating: pods "kube-auth-proxy-cb779b95-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-vtsgd

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

kube-auth-proxy-cb779b95-ww4qf

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-ww4qf

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-cb779b95-vtsgd

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-b2j7q

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-b2j7q

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-ww4qf

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-ww4qf

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-ww4qf

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.706s (2.706s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

Created

Created container: discovery

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-vtsgd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 6.057s (6.057s including waiting). Image size: 265120132 bytes.

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-vkhlm

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-vtsgd

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-cb779b95-vtsgd

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 4.865s (4.865s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

Started

Started container discovery

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-vkhlm

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-vkhlm

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-vkhlm

Started

Started container discovery

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-68cf4d799c from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-vkhlm

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.01s (2.011s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-vkhlm

Created

Created container: discovery

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-68cf4d799c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-vkhlm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-vkhlm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-b2j7q

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-b2j7q

Killing

Stopping container discovery

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.729s (2.729s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799ch6bdd

Created

Created container: istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5ccb968c66

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5ccb968c66-7kkbk

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5ccb968c66 from 0 to 1

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

multus

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-wkmjz

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-wkmjz

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-ncl9j

openshift-ingress

multus

payload-processing-5b54b7fbb7-ncl9j

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ncl9j

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-wkmjz

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ncl9j

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.18s (3.18s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ncl9j

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ncl9j

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-wkmjz

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.22s (3.22s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-wkmjz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-wkmjz

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

Unhealthy

Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-7kkbk

ProbeError

Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-worker-w4-1910e9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-1910e9-openshift-default-68f655766 from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w0-85d3ec-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-85d3ec-openshift-default-6f568cc64d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-85d3ec

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-85d3ec

openshift-ingress

replicaset-controller

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d

SuccessfulCreate

Created pod: e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

openshift-ingress

multus

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-inf-a-d36f67

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d36f67

openshift-ingress

deployment-controller

e2e-inf-a-d36f67-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-d36f67-openshift-default-66569bd74b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-1910e9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-1910e9

openshift-ingress

replicaset-controller

e2e-inf-a-d36f67-openshift-default-66569bd74b

SuccessfulCreate

Created pod: e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

openshift-ingress

replicaset-controller

e2e-worker-w4-1910e9-openshift-default-68f655766

SuccessfulCreate

Created pod: e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

openshift-ingress

multus

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-d36f67

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-d36f67-57d968c98b from 0 to 1

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 3.862s (3.862s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-d36f67

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-d36f67-58cc6b6469 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-85d3ec

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-85d3ec-68799dcc5f from 0 to 1

openshift-ingress

multus

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-d36f67-58cc6b6469

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-d36f67-58cc6b6469-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d36f67: serviceaccount "payload-processing-e2e-inf-a-d36f67" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-d36f67-58cc6b6469

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

Started

Started container payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-85d3ec

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-85d3ec: serviceaccount "payload-processing-e2e-worker-w0-85d3ec" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-85d3ec: serviceaccount "payload-processing-e2e-worker-w0-85d3ec" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d36f67: serviceaccount "payload-processing-e2e-inf-a-d36f67" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

openshift-ingress

multus

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.292s (1.292s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.317s (1.317s including waiting). Image size: 166449312 bytes.

openshift-ingress

deployment-controller

e2e-autores-30eaba-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-30eaba-openshift-default-7679f8495d from 0 to 1

openshift-ingress

replicaset-controller

e2e-autores-30eaba-openshift-default-7679f8495d

SuccessfulCreate

Created pod: e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

openshift-ingress

gateway_labeler_controller

e2e-autores-30eaba

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-30eaba

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

Started

Started container payload-pre-processing

openshift-ingress

multus

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

Created

Created container: payload-processing

openshift-ingress

multus

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ait-3be523b2-openshift-default-5c76fd8768

SuccessfulCreate

Created pod: e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

gateway_labeler_controller

e2e-ait-3be523b2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-3be523b2

openshift-ingress

deployment-controller

e2e-ait-3be523b2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-3be523b2-openshift-default-5c76fd8768 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-1910e9: serviceaccount "payload-processing-e2e-worker-w4-1910e9" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-1910e9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-1910e9-7f75599d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-1910e9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-1910e9: serviceaccount "payload-processing-e2e-worker-w4-1910e9" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-30eaba-5f56c9b6d5

FailedCreate

Error creating: pods "payload-processing-e2e-autores-30eaba-5f56c9b6d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-30eaba: serviceaccount "payload-processing-e2e-autores-30eaba" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 4.134s (4.134s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.124s (1.124s including waiting). Image size: 166449312 bytes.

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-30eaba

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-30eaba-56bb688b6b from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-autores-30eaba

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-30eaba-5f56c9b6d5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-30eaba-5f56c9b6d5

SuccessfulCreate

Created pod: payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-30eaba-56bb688b6b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-30eaba-56bb688b6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-30eaba: serviceaccount "payload-processing-e2e-autores-30eaba" not found

openshift-ingress

multus

payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-30eaba-56bb688b6b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

openshift-ingress

kubelet

payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-inf-b-e9cc98

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-e9cc98

openshift-ingress

multus

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

AddedInterface

Add eth0 [10.132.0.60/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-inf-b-e9cc98-openshift-default-84976fb6d6

SuccessfulCreate

Created pod: e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

openshift-ingress

deployment-controller

e2e-inf-b-e9cc98-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-e9cc98-openshift-default-84976fb6d6 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-3be523b2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-3be523b2-547cbc54d7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-3be523b2-547cbc54d7

SuccessfulCreate

Created pod: payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-3be523b2-547cbc54d7

FailedCreate

Error creating: pods "payload-processing-e2e-ait-3be523b2-547cbc54d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-3be523b2: serviceaccount "payload-processing-e2e-ait-3be523b2" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-3be523b2: serviceaccount "payload-processing-e2e-ait-3be523b2" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-3be523b2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-e9cc98

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-e9cc98

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e9cc98: serviceaccount "payload-processing-e2e-inf-b-e9cc98" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e9cc98: serviceaccount "payload-processing-e2e-inf-b-e9cc98" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

Started

Started container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

Unhealthy

Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

ProbeError

Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

ProbeError

Readiness probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Unhealthy

Readiness probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Unhealthy

Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

ProbeError

Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

ProbeError

Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Unhealthy

Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-3be523b2-openshift-default-5c76fd8768-jdnnz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ait-3be523b2-547cbc54d7-jjvhq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-3be523b2-6d457d8fdb-54zlm

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-3be523b2

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-3be523b2: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-3be523b2": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-3be523b2, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 1d550ca3-706c-4896-b9dd-31cf7b92dcd9, UID in object meta:

openshift-ingress

replicaset-controller

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4

SuccessfulCreate

Created pod: e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

openshift-ingress

deployment-controller

e2e-migrate-77bab00f-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-77bab00f-gw-openshift-default-845565d4b4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-77bab00f-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-77bab00f-gw

openshift-ingress

multus

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

ProbeError

Readiness probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Unhealthy

Readiness probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Created

Created container: istio-proxy
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-77bab00f: serviceaccount "payload-processing-e2e-migrate-77bab00f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-77bab00f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-77bab00f-d5f869458 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-77bab00f-d5f869458

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-77bab00f-d5f869458

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-77bab00f-d5f869458-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-77bab00f: serviceaccount "payload-processing-e2e-migrate-77bab00f" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-77bab00f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

openshift-ingress

kubelet

payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Unhealthy

Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

ProbeError

Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

kubelet

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

Unhealthy

Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

ProbeError

Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-migrate-77bab00f-d5f869458-jt8rt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-77bab00f-65685fbb8b-8j4zb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-77bab00f-gw-openshift-default-845565d4b4-ls884

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ait-18c04da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-18c04da2-openshift-default-555c79855d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-18c04da2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-18c04da2

openshift-ingress

replicaset-controller

e2e-ait-18c04da2-openshift-default-555c79855d

SuccessfulCreate

Created pod: e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

openshift-ingress

multus

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-18c04da2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-18c04da2-7b7f85b765 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-18c04da2-7b7f85b765

SuccessfulCreate

Created pod: payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-18c04da2-7b7f85b765

FailedCreate

Error creating: pods "payload-processing-e2e-ait-18c04da2-7b7f85b765-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-18c04da2: serviceaccount "payload-processing-e2e-ait-18c04da2" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-18c04da2-647697cf94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-18c04da2-647697cf94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-18c04da2: serviceaccount "payload-processing-e2e-ait-18c04da2" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-18c04da2-647697cf94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-18c04da2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-18c04da2-647697cf94 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

Unhealthy

Startup probe failed: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

ProbeError

Startup probe error: Get "http://10.134.0.47:15021/healthz/ready": dial tcp 10.134.0.47:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-18c04da2

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-18c04da2: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-18c04da2": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-18c04da2, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 9b6c0996-941e-4043-86dd-1ecb8feb0b16, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-18c04da2-647697cf94-t7jx8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-18c04da2-7b7f85b765-x27rg

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-derive-c62bec51-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-c62bec51-openshift-default-585f889dfb from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-c62bec51-openshift-default-585f889dfb

SuccessfulCreate

Created pod: e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

openshift-ingress

gateway_labeler_controller

e2e-derive-c62bec51

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-c62bec51

openshift-ingress

kubelet

e2e-ait-18c04da2-openshift-default-555c79855d-vth4r

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-derive-c62bec51

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-c62bec51-59c8545577 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-c62bec51-6677694ff4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-c62bec51-6677694ff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c62bec51: serviceaccount "payload-processing-e2e-derive-c62bec51" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-c62bec51-59c8545577

SuccessfulCreate

Created pod: payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-c62bec51-59c8545577

FailedCreate

Error creating: pods "payload-processing-e2e-derive-c62bec51-59c8545577-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c62bec51: serviceaccount "payload-processing-e2e-derive-c62bec51" not found

openshift-ingress

kubelet

payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-c62bec51

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-c62bec51-6677694ff4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-c62bec51-6677694ff4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

ProbeError

Readiness probe error: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Unhealthy

Readiness probe failed: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

ProbeError

Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Unhealthy

Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-c62bec51-6677694ff4-5x6q6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-c62bec51-openshift-default-585f889dfb-mx7jm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-30eaba-56bb688b6b-tdlmf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-c62bec51-59c8545577-rmgzq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-30eaba-5f56c9b6d5-97rkm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-explicit-cc7bcd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6

SuccessfulCreate

Created pod: e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

openshift-ingress

gateway_labeler_controller

e2e-explicit-cc7bcd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-cc7bcd

openshift-ingress

multus

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

Unhealthy

Readiness probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused

openshift-ingress

kubelet

e2e-autores-30eaba-openshift-default-7679f8495d-bfmlc

ProbeError

Readiness probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-cc7bcd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-cc7bcd-68cc856686 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-cc7bcd: serviceaccount "payload-processing-e2e-explicit-cc7bcd" not found

openshift-ingress

multus

payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-cc7bcd-f48d7657

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-cc7bcd-f48d7657-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-cc7bcd: serviceaccount "payload-processing-e2e-explicit-cc7bcd" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-cc7bcd-f48d7657

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-cc7bcd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-cc7bcd-f48d7657 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Unhealthy

Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

ProbeError

Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-mt-34925ca5-openshift-default-7d4b568588

SuccessfulCreate

Created pod: e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

openshift-ingress

deployment-controller

e2e-mt-34925ca5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-34925ca5-openshift-default-7d4b568588 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-34925ca5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-34925ca5

openshift-ingress

multus

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-34925ca5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-34925ca5-5fcffdc7b9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-34925ca5-5fcffdc7b9

SuccessfulCreate

Created pod: payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-34925ca5: serviceaccount "payload-processing-e2e-mt-34925ca5" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-34925ca5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-34925ca5-5fcffdc7b9

FailedCreate

Error creating: pods "payload-processing-e2e-mt-34925ca5-5fcffdc7b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-34925ca5: serviceaccount "payload-processing-e2e-mt-34925ca5" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-explicit-cc7bcd-f48d7657-8gg9v

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-explicit-cc7bcd

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-explicit-cc7bcd: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-explicit-cc7bcd": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-explicit-cc7bcd, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 76ffbffc-efb6-4440-a486-28b1aa9e3fe2, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-cc7bcd-68cc856686-rvmsg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-34925ca5-5dd455bcc8-mhhd8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-34925ca5-5fcffdc7b9-6php7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-noten-f68032-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-f68032-openshift-default-6f79c44bc8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-noten-f68032-openshift-default-6f79c44bc8

SuccessfulCreate

Created pod: e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

openshift-ingress

gateway_labeler_controller

e2e-noten-f68032

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-f68032

openshift-ingress

kubelet

e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-explicit-cc7bcd-openshift-default-5bfbbbfdc6-vdgbg

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-34925ca5-openshift-default-7d4b568588-qzjbw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-shared-a-w2-3503d7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-3503d7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-3503d7

openshift-ingress

replicaset-controller

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d

SuccessfulCreate

Created pod: e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

openshift-ingress

multus

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-3503d7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-3503d7: serviceaccount "payload-processing-e2e-shared-a-w2-3503d7" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-3503d7: serviceaccount "payload-processing-e2e-shared-a-w2-3503d7" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-3503d7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Started

Started container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-558928

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-558928

openshift-ingress

replicaset-controller

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd

SuccessfulCreate

Created pod: e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

openshift-ingress

deployment-controller

e2e-shared-b-w2-558928-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-558928-openshift-default-6b7b75cffd from 0 to 1

openshift-ingress

multus

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-d36f67-57d968c98b-mdc6m

Unhealthy

Readiness probe failed: dial tcp 10.132.0.56:9004: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-d36f67-58cc6b6469-8nvzl

Unhealthy

Readiness probe failed: dial tcp 10.132.0.57:9004: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-558928: serviceaccount "payload-processing-e2e-shared-b-w2-558928" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-558928-5644fb48f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-558928-5644fb48f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-558928: serviceaccount "payload-processing-e2e-shared-b-w2-558928" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-558928-5644fb48f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-558928

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-558928-5644fb48f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-558928

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-558928-58b676c498 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-d36f67-openshift-default-66569bd74b-pfmlj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e9cc98-dd8f46fcd-m2bd6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e9cc98-5c6d7ccc65-njnvj

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

Unhealthy

Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

ProbeError

Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-1cbcd0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-1cbcd0

openshift-ingress

deployment-controller

e2e-shared-a-w3-1cbcd0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d

SuccessfulCreate

Created pod: e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

openshift-ingress

kubelet

e2e-noten-f68032-openshift-default-6f79c44bc8-hdnmm

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-mt-5bbd7765

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-5bbd7765

openshift-ingress

deployment-controller

e2e-mt-5bbd7765-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-5bbd7765-openshift-default-7b87658c64 from 0 to 1

openshift-ingress

multus

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-5bbd7765-openshift-default-7b87658c64

SuccessfulCreate

Created pod: e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-inf-b-e9cc98-openshift-default-84976fb6d6-ptq57

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-1cbcd0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-1cbcd0: serviceaccount "payload-processing-e2e-shared-a-w3-1cbcd0" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-1cbcd0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-1cbcd0: serviceaccount "payload-processing-e2e-shared-a-w3-1cbcd0" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt

Started

Started container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-mt-5bbd7765

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-5bbd7765-5569b7ff9c from 0 to 1

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-5bbd7765-5569b7ff9c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-5bbd7765-5569b7ff9c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-5bbd7765-5569b7ff9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-5bbd7765: serviceaccount "payload-processing-e2e-mt-5bbd7765" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-5bbd7765

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-5bbd7765-977855946 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-5bbd7765-977855946

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-5bbd7765-977855946

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-5bbd7765-977855946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-5bbd7765: serviceaccount "payload-processing-e2e-mt-5bbd7765" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd

SuccessfulCreate

Created pod: e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

openshift-ingress

deployment-controller

e2e-shared-b-w3-7bdcda-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-7bdcda

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-7bdcda

openshift-ingress

multus

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

Unhealthy

Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

ProbeError

Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-7bdcda: serviceaccount "payload-processing-e2e-shared-b-w3-7bdcda" not found

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-7bdcda: serviceaccount "payload-processing-e2e-shared-b-w3-7bdcda" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-7bdcda

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-7bdcda

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Unhealthy

Readiness probe failed: Get "http://10.134.0.67:15021/healthz/ready": read tcp 10.134.0.2:44780->10.134.0.67:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

ProbeError

Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": read tcp 10.134.0.2:44780->10.134.0.67:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Unhealthy

Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

ProbeError

Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-85d3ec-68799dcc5f-n8j5h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-85d3ec-7c96c86b66-zcbqp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w0-85d3ec-openshift-default-6f568cc64d-t6pfq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-mt-5bbd7765-5569b7ff9c-wpf7h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5bbd7765-977855946-fd96q

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-5bbd7765-openshift-default-7b87658c64-l67kt

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-api-a-d220da-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-d220da-openshift-default-74b6789978 from 0 to 1

openshift-ingress

multus

e2e-api-a-d220da-openshift-default-74b6789978-29pww

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-api-a-d220da

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d220da

openshift-ingress

replicaset-controller

e2e-api-a-d220da-openshift-default-74b6789978

SuccessfulCreate

Created pod: e2e-api-a-d220da-openshift-default-74b6789978-29pww
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-d220da-b9bb86c45

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-d220da-b9bb86c45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d220da: serviceaccount "payload-processing-e2e-api-a-d220da" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-d220da

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-d220da-b9bb86c45 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-d220da-b9bb86c45

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-d220da

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-d220da-6d95b5c654 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-d220da-6d95b5c654

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-d220da-6d95b5c654

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-d220da-6d95b5c654-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d220da: serviceaccount "payload-processing-e2e-api-a-d220da" not found

openshift-ingress

kubelet

payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

AddedInterface

Add eth0 [10.132.0.93/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

Created

Created container: payload-processing

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Unhealthy

Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

ProbeError

Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

replicaset-controller

e2e-api-b-ea2835-openshift-default-6d5787d89d

SuccessfulCreate

Created pod: e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

openshift-ingress

gateway_labeler_controller

e2e-api-b-ea2835

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-ea2835

openshift-ingress

deployment-controller

e2e-api-b-ea2835-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-ea2835-openshift-default-6d5787d89d from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Created

Created container: istio-proxy

openshift-ingress

multus

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-ea2835

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-ea2835-59746c6f6f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-ea2835

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-ea2835-7c65d87654 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-ea2835-7c65d87654

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-ea2835-7c65d87654-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ea2835: serviceaccount "payload-processing-e2e-api-b-ea2835" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-ea2835-59746c6f6f

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-ea2835-59746c6f6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ea2835: serviceaccount "payload-processing-e2e-api-b-ea2835" not found

openshift-ingress

kubelet

payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

AddedInterface

Add eth0 [10.133.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-ea2835-7c65d87654

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-ea2835-59746c6f6f

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

AddedInterface

Add eth0 [10.132.0.94/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

Unhealthy

Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

ProbeError

Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-d220da-b9bb86c45-59pzn

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-a-d220da

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-a-d220da: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-a-d220da": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-a-d220da, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 03eb7eb2-4c9c-4eec-98f5-c12839ea9f67, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-d220da-6d95b5c654-sq8bt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-a-d220da-openshift-default-74b6789978-29pww

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-1910e9-7f75599d-2klsw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-1910e9-5fdc8d8bd5-zvjfh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-ea2835-7c65d87654-v4zs6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-ea2835-59746c6f6f-zh478

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w4-1910e9-openshift-default-68f655766-s56th

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-ea2835-openshift-default-6d5787d89d-zhn5z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3503d7-74ccdc868f-gtds5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3503d7-7855d9574b-pq4cl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-a-w2-3503d7-openshift-default-5f79cd469d-5hfx8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-558928-5644fb48f-jflh8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-558928-58b676c498-pz8hf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-558928-openshift-default-6b7b75cffd-dspv8

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ipp-a-689fac-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-689fac-openshift-default-d8d45697b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-689fac-openshift-default-d8d45697b

SuccessfulCreate

Created pod: e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-689fac

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-689fac

openshift-ingress

multus

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-689fac-79754fc76c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-689fac-79754fc76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-689fac: serviceaccount "payload-processing-e2e-ipp-a-689fac" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-689fac

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-689fac-79754fc76c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-689fac-79754fc76c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-689fac

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-689fac: serviceaccount "payload-processing-e2e-ipp-a-689fac" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-7406d4

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-7406d4

openshift-ingress

replicaset-controller

e2e-ipp-b-7406d4-openshift-default-79fbf5898

SuccessfulCreate

Created pod: e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

openshift-ingress

deployment-controller

e2e-ipp-b-7406d4-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-7406d4-openshift-default-79fbf5898 from 0 to 1

openshift-ingress

multus

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

AddedInterface

Add eth0 [10.134.0.82/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-7406d4

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-7406d4-859f7cf49 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-7406d4

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7406d4: serviceaccount "payload-processing-e2e-ipp-b-7406d4" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-7406d4-859f7cf49

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-7406d4-859f7cf49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7406d4: serviceaccount "payload-processing-e2e-ipp-b-7406d4" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-7406d4-859f7cf49

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

Started

Started container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

ProbeError

Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Unhealthy

Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

ProbeError

Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Unhealthy

Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4

SuccessfulCreate

Created pod: e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-7599d7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-7599d7

openshift-ingress

deployment-controller

e2e-ipp-cleanup-7599d7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4 from 0 to 1

openshift-ingress

multus

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

AddedInterface

Add eth0 [10.134.0.85/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-7599d7: serviceaccount "payload-processing-e2e-ipp-cleanup-7599d7" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-7599d7: serviceaccount "payload-processing-e2e-ipp-cleanup-7599d7" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-7599d7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-7599d7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : failed to sync configmap cache: timed out waiting for the condition

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

AddedInterface

Add eth0 [10.132.0.103/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

ProbeError

Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

Unhealthy

Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-7599d7-7974d5f678-x5rcg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-7599d7-5679466c8f-nds9f

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-7599d7-openshift-default-858c8888c4-8pdl4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-689fac-557f44bdfd-vsm96

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-689fac-79754fc76c-p4r4b

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-a-689fac-openshift-default-d8d45697b-hfcss

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-7406d4-6d959666c8-9826v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-7406d4-859f7cf49-mxr5d

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-7406d4-openshift-default-79fbf5898-j2jxh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-1cbcd0-5dbd87cccc-m68h6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-1cbcd0-6755ccc4db-sh2xt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-1cbcd0-openshift-default-5f656d4c9d-hptg9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-7bdcda-7d4d7d4c75-mm46w

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-7bdcda-644fb84d54-wnvts

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w3-7bdcda-openshift-default-549b4b9bbd-fr9hz

Killing

Stopping container istio-proxy