Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

Scheduled

Successfully assigned openshift-ingress/e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Scheduled

Successfully assigned openshift-ingress/e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt to ip-10-0-142-22.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-86c464-9c9b87947-pcvkm to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-derive-2f37280d-57dd94865-k252b

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-2f37280d-57dd94865-k252b to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c to ip-10-0-142-22.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Scheduled

Successfully assigned openshift-ingress/e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq to ip-10-0-142-22.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-68p88

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-68p88 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-080801-openshift-default-84855b7986-dx695 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Scheduled

Successfully assigned openshift-ingress/e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd to ip-10-0-142-22.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5 to ip-10-0-142-22.ec2.internal

openshift-ingress

kube-auth-proxy-549869464b-8trnz

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-549869464b-8trnz to ip-10-0-142-22.ec2.internal

openshift-ingress

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

Scheduled

Successfully assigned openshift-ingress/e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz to ip-10-0-128-175.ec2.internal

openshift-ingress

kube-auth-proxy-549869464b-7pvhb

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-549869464b-7pvhb to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q to ip-10-0-128-175.ec2.internal

openshift-ingress

router-default-7d847c5fc8-txhzg

Scheduled

Successfully assigned openshift-ingress/router-default-7d847c5fc8-txhzg to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j to ip-10-0-142-22.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2 to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77 to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

Scheduled

Successfully assigned openshift-ingress/e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-fkwcw

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-fkwcw to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-q95h6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-q95h6 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt to ip-10-0-142-22.ec2.internal

openshift-ingress

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6 to ip-10-0-128-175.ec2.internal

openshift-ingress

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2 to ip-10-0-128-175.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-2fmw5

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-2fmw5 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb to ip-10-0-128-175.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4 to ip-10-0-133-98.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq to ip-10-0-133-98.ec2.internal

openshift-ingress

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Scheduled

Successfully assigned openshift-ingress/e2e-derive-2f37280d-openshift-default-647575f4d-btg4c to ip-10-0-128-175.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-4x7rq to ip-10-0-128-175.ec2.internal

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-7d847c5fc8 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

router-default-7d847c5fc8

FailedCreate

Error creating: pods "router-default-7d847c5fc8-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-7d847c5fc8

FailedCreate

Error creating: pods "router-default-7d847c5fc8-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-7d847c5fc8

SuccessfulCreate

Created pod: router-default-7d847c5fc8-txhzg
(x5)

openshift-ingress

kubelet

router-default-7d847c5fc8-txhzg

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-7d847c5fc8-txhzg

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-7d847c5fc8-txhzg

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

multus

router-default-7d847c5fc8-txhzg

AddedInterface

Add eth0 [10.134.0.12/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-7d847c5fc8-txhzg

Started

Started container router

openshift-ingress

kubelet

router-default-7d847c5fc8-txhzg

Created

Created container: router

openshift-ingress

replicaset-controller

kube-auth-proxy-549869464b

SuccessfulCreate

Created pod: kube-auth-proxy-549869464b-8trnz

openshift-ingress

replicaset-controller

kube-auth-proxy-549869464b

SuccessfulCreate

Created pod: kube-auth-proxy-549869464b-7pvhb
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-549869464b

FailedCreate

Error creating: pods "kube-auth-proxy-549869464b-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-549869464b-8trnz

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-549869464b from 0 to 2

openshift-ingress

kubelet

kube-auth-proxy-549869464b-7pvhb

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

multus

kube-auth-proxy-549869464b-7pvhb

AddedInterface

Add eth0 [10.132.0.15/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-549869464b-7pvhb

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-549869464b-8trnz

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-549869464b-8trnz

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-68p88

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-68p88

AddedInterface

Add eth0 [10.132.0.16/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-549869464b-7pvhb

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-549869464b-8trnz

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-549869464b-7pvhb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.74s (2.74s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-549869464b-8trnz

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.703s (2.703s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-549869464b-7pvhb

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-549869464b-8trnz

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.323s (3.323s including waiting). Image size: 181126104 bytes.

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-hl7q2

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.385s (2.385s including waiting). Image size: 181318953 bytes.

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-2fmw5

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-2fmw5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" already present on machine

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-2fmw5

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-2fmw5

Created

Created container: discovery

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-2fmw5

AddedInterface

Add eth0 [10.132.0.19/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-2fmw5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-2fmw5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-68p88

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-68p88

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-4x7rq

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-fkwcw

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-q95h6

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-q95h6

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-q95h6

AddedInterface

Add eth0 [10.132.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-fkwcw

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-5b54b7fbb7-fkwcw

AddedInterface

Add eth0 [10.132.0.32/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-q95h6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.481s (1.481s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-fkwcw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.482s (1.482s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-q95h6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-fkwcw

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-fkwcw

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-q95h6

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

Unhealthy

Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-4x7rq

ProbeError

Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-080801

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-080801

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-8b50f2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-8b50f2

openshift-ingress

replicaset-controller

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b

SuccessfulCreate

Created pod: e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

openshift-ingress

deployment-controller

e2e-inf-a-2e1b82-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-a-2e1b82

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-2e1b82

openshift-ingress

deployment-controller

e2e-worker-w0-080801-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-080801-openshift-default-84855b7986 from 0 to 1

openshift-ingress

multus

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-8b50f2-openshift-default-869f86786

SuccessfulCreate

Created pod: e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

openshift-ingress

deployment-controller

e2e-worker-w4-8b50f2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-8b50f2-openshift-default-869f86786 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w0-080801-openshift-default-84855b7986

SuccessfulCreate

Created pod: e2e-worker-w0-080801-openshift-default-84855b7986-dx695

openshift-ingress

multus

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

multus

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-080801-6df746bf4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-080801-6df746bf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-080801: serviceaccount "payload-processing-e2e-worker-w0-080801" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-080801

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-080801-84db66fd65 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-080801-84db66fd65

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-080801-84db66fd65

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-080801-84db66fd65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-080801: serviceaccount "payload-processing-e2e-worker-w0-080801" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-080801

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-080801-6df746bf4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-080801-6df746bf4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-8b50f2: serviceaccount "payload-processing-e2e-worker-w4-8b50f2" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-8b50f2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-8b50f2-77b8c4548f from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-8b50f2: serviceaccount "payload-processing-e2e-worker-w4-8b50f2" not found

openshift-ingress

multus

payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-8b50f2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

Started

Started container payload-pre-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-2e1b82

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-2e1b82

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f from 0 to 1

openshift-ingress

multus

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

multus

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

openshift-ingress

gateway_labeler_controller

e2e-autores-86c464

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-86c464

openshift-ingress

deployment-controller

e2e-autores-86c464-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-86c464-openshift-default-5d966fdd44 from 0 to 1

openshift-ingress

replicaset-controller

e2e-autores-86c464-openshift-default-5d966fdd44

SuccessfulCreate

Created pod: e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2e1b82: serviceaccount "payload-processing-e2e-inf-a-2e1b82" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2e1b82: serviceaccount "payload-processing-e2e-inf-a-2e1b82" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.622s (1.622s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 415ms (415ms including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

Started

Started container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-a48a4ae7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a48a4ae7

openshift-ingress

GatewayAuthBootstrap

e2e-ait-a48a4ae7

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ait-a48a4ae7-authn-ssl: envoyfilters.networking.istio.io "e2e-ait-a48a4ae7-authn-ssl" already exists

openshift-ingress

multus

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ait-a48a4ae7-openshift-default-778678f8db

SuccessfulCreate

Created pod: e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

openshift-ingress

deployment-controller

e2e-ait-a48a4ae7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-a48a4ae7-openshift-default-778678f8db from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-86c464-9c9b87947

FailedCreate

Error creating: pods "payload-processing-e2e-autores-86c464-9c9b87947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-86c464: serviceaccount "payload-processing-e2e-autores-86c464" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-86c464-55dd94bb94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-86c464-55dd94bb94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-86c464: serviceaccount "payload-processing-e2e-autores-86c464" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-86c464-55dd94bb94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

openshift-ingress

deployment-controller

payload-processing-e2e-autores-86c464

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-86c464-9c9b87947 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-86c464

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-86c464-55dd94bb94 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-86c464-9c9b87947

SuccessfulCreate

Created pod: payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

openshift-ingress

multus

payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-a48a4ae7-7569987ddf

FailedCreate

Error creating: pods "payload-processing-e2e-ait-a48a4ae7-7569987ddf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a48a4ae7: serviceaccount "payload-processing-e2e-ait-a48a4ae7" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-a48a4ae7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a48a4ae7: serviceaccount "payload-processing-e2e-ait-a48a4ae7" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

Started

Started container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-ait-a48a4ae7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-a48a4ae7-7569987ddf from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-a48a4ae7-7569987ddf

SuccessfulCreate

Created pod: payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.758s (3.758s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

multus

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-inf-b-28f011-openshift-default-65dbc584cb

SuccessfulCreate

Created pod: e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q
(x2)

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Started

Started container istio-proxy

openshift-ingress

deployment-controller

e2e-inf-b-28f011-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-28f011-openshift-default-65dbc584cb from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

gateway_labeler_controller

e2e-inf-b-28f011

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-28f011
(x2)

openshift-ingress

kubelet

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

Started

Started container istio-proxy

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-28f011

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-28f011

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-28f011-6bcd7f4887 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-28f011: serviceaccount "payload-processing-e2e-inf-b-28f011" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-28f011-6bcd7f4887

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-28f011-6bcd7f4887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-28f011: serviceaccount "payload-processing-e2e-inf-b-28f011" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-28f011-6bcd7f4887

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt
(x8)

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

ProbeError

Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Unhealthy

Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

Unhealthy

Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

ProbeError

Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

Unhealthy

Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

ProbeError

Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-a48a4ae7-5c57d5f79d-ghx7c

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ait-a48a4ae7-7569987ddf-7gbn9

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

ProbeError

Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-migrate-7f7de479-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-7f7de479-gw

openshift-ingress

replicaset-controller

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d

SuccessfulCreate

Created pod: e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

openshift-ingress

deployment-controller

e2e-migrate-7f7de479-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

Unhealthy

Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused

openshift-ingress

multus

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-a48a4ae7-openshift-default-778678f8db-2hgh7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-7f7de479

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-7f7de479: serviceaccount "payload-processing-e2e-migrate-7f7de479" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-7f7de479: serviceaccount "payload-processing-e2e-migrate-7f7de479" not found

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-7f7de479

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-7f7de479-7fddcb7f7d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : failed to sync configmap cache: timed out waiting for the condition

openshift-ingress

kubelet

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : failed to sync configmap cache: timed out waiting for the condition

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

ProbeError

Startup probe error: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Unhealthy

Startup probe failed: Get "http://10.134.0.48:15021/healthz/ready": dial tcp 10.134.0.48:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-migrate-7f7de479-7fddcb7f7d-v2blr

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-migrate-7f7de479

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-migrate-7f7de479: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-migrate-7f7de479": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-migrate-7f7de479, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6c2d0429-8f45-4248-9565-07bf9ed588dc, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-7f7de479-84b8898c9d-6tnhh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ait-1ecaca94-openshift-default-b9c7455fd

SuccessfulCreate

Created pod: e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

openshift-ingress

GatewayAuthBootstrap

e2e-ait-1ecaca94

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ait-1ecaca94-authn-ssl: envoyfilters.networking.istio.io "e2e-ait-1ecaca94-authn-ssl" already exists

openshift-ingress

multus

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ait-1ecaca94

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1ecaca94

openshift-ingress

deployment-controller

e2e-ait-1ecaca94-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-1ecaca94-openshift-default-b9c7455fd from 0 to 1

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-migrate-7f7de479-gw-openshift-default-6d45fb6c4d-th9k5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

payload-processing-e2e-ait-1ecaca94

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-1ecaca94-656f8d7c4f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-1ecaca94-656f8d7c4f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-1ecaca94

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1ecaca94: serviceaccount "payload-processing-e2e-ait-1ecaca94" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-1ecaca94-656f8d7c4f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-1ecaca94-656f8d7c4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1ecaca94: serviceaccount "payload-processing-e2e-ait-1ecaca94" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ecaca94-7879cd6b94-7flzd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-1ecaca94-656f8d7c4f-qzjdb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-2f37280d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-2f37280d-openshift-default-647575f4d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-2f37280d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-2f37280d

openshift-ingress

replicaset-controller

e2e-derive-2f37280d-openshift-default-647575f4d

SuccessfulCreate

Created pod: e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

openshift-ingress

kubelet

payload-processing-e2e-autores-86c464-9c9b87947-pcvkm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-86c464-55dd94bb94-6pjvd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-1ecaca94-openshift-default-b9c7455fd-n6q48

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

payload-processing-e2e-derive-2f37280d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-2f37280d-57dd94865 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-2f37280d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-2f37280d-57dd94865

FailedCreate

Error creating: pods "payload-processing-e2e-derive-2f37280d-57dd94865-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-2f37280d: serviceaccount "payload-processing-e2e-derive-2f37280d" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-2f37280d: serviceaccount "payload-processing-e2e-derive-2f37280d" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-2f37280d-57dd94865

SuccessfulCreate

Created pod: payload-processing-e2e-derive-2f37280d-57dd94865-k252b

openshift-ingress

kubelet

payload-processing-e2e-derive-2f37280d-57dd94865-k252b

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-2f37280d-57dd94865-k252b

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-derive-2f37280d-57dd94865-k252b

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-2f37280d-57dd94865-k252b

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Unhealthy

Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

ProbeError

Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-2f37280d-6dd7d5cfbb-kbgqz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-2f37280d-57dd94865-k252b

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-derive-2f37280d-openshift-default-647575f4d-btg4c

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-autores-86c464-openshift-default-5d966fdd44-qlqvk

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-explicit-0e6aaa-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-0e6aaa-openshift-default-6c7f796c8d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-explicit-0e6aaa

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-0e6aaa

openshift-ingress

multus

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d

SuccessfulCreate

Created pod: e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-0e6aaa

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-0e6aaa-fdc589f45 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-0e6aaa-fdc589f45

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-0e6aaa-fdc589f45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-0e6aaa: serviceaccount "payload-processing-e2e-explicit-0e6aaa" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-0e6aaa

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-0e6aaa: serviceaccount "payload-processing-e2e-explicit-0e6aaa" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-0e6aaa-fdc589f45

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

ProbeError

Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

Unhealthy

Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-mt-130b7aa1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-130b7aa1

openshift-ingress

multus

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-130b7aa1-openshift-default-7855464968

SuccessfulCreate

Created pod: e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

openshift-ingress

deployment-controller

e2e-mt-130b7aa1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-130b7aa1-openshift-default-7855464968 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-explicit-0e6aaa-fdc589f45-ltq9h

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0e6aaa-6bcb96874b-n2jl7

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-130b7aa1: serviceaccount "payload-processing-e2e-mt-130b7aa1" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-130b7aa1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-130b7aa1-7db599d5cc

FailedCreate

Error creating: pods "payload-processing-e2e-mt-130b7aa1-7db599d5cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-130b7aa1: serviceaccount "payload-processing-e2e-mt-130b7aa1" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-130b7aa1-7db599d5cc

SuccessfulCreate

Created pod: payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

openshift-ingress

deployment-controller

payload-processing-e2e-mt-130b7aa1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-130b7aa1-7db599d5cc from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-explicit-0e6aaa-openshift-default-6c7f796c8d-x65dl

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-noten-adae21-openshift-default-68bdbc5fb

SuccessfulCreate

Created pod: e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

openshift-ingress

gateway_labeler_controller

e2e-noten-adae21

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-adae21

openshift-ingress

deployment-controller

e2e-noten-adae21-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-adae21-openshift-default-68bdbc5fb from 0 to 1

openshift-ingress

multus

e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

ProbeError

Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

Unhealthy

Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-mt-130b7aa1-7db599d5cc-l6zgj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-130b7aa1-6855bb4f98-96kkb

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-130b7aa1

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-130b7aa1: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-130b7aa1": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-130b7aa1, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 67645992-1088-4934-adf5-ca25dd9051d5, UID in object meta:

openshift-ingress

kubelet

e2e-mt-130b7aa1-openshift-default-7855464968-kqdt7

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-4009da

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-4009da

openshift-ingress

deployment-controller

e2e-shared-a-w2-4009da-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-4009da-openshift-default-76fc699c86 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w2-4009da-openshift-default-76fc699c86

SuccessfulCreate

Created pod: e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

openshift-ingress

multus

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-4009da

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-4009da

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-4009da: serviceaccount "payload-processing-e2e-shared-a-w2-4009da" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-4009da: serviceaccount "payload-processing-e2e-shared-a-w2-4009da" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

Created

Created container: payload-processing

openshift-ingress

kubelet

e2e-noten-adae21-openshift-default-68bdbc5fb-fd7xk

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-60bf73

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-60bf73

openshift-ingress

deployment-controller

e2e-shared-b-w2-60bf73-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-60bf73-openshift-default-66966687fb from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w2-60bf73-openshift-default-66966687fb

SuccessfulCreate

Created pod: e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

openshift-ingress

multus

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

Created

Created container: istio-proxy
(x7)

openshift-ingress

kubelet

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

ProbeError

Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

Unhealthy

Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-60bf73

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-60bf73

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-60bf73: serviceaccount "payload-processing-e2e-shared-b-w2-60bf73" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-60bf73: serviceaccount "payload-processing-e2e-shared-b-w2-60bf73" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-2e1b82-7d5dcc7b4f-nnbrq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-2e1b82-844cfcbc9d-rqlhq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-a-2e1b82-openshift-default-5f94d9fd8b-pvsvs

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-mt-d2d17aff

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d2d17aff

openshift-ingress

deployment-controller

e2e-mt-d2d17aff-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-d2d17aff-openshift-default-74687dc448 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-d2d17aff-openshift-default-74687dc448

SuccessfulCreate

Created pod: e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

openshift-ingress

multus

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

ProbeError

Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

Unhealthy

Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d2d17aff: serviceaccount "payload-processing-e2e-mt-d2d17aff" not found

openshift-ingress

deployment-controller

payload-processing-e2e-mt-d2d17aff

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-d2d17aff-676cd69d4f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d2d17aff-676cd69d4f

FailedCreate

Error creating: pods "payload-processing-e2e-mt-d2d17aff-676cd69d4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d2d17aff: serviceaccount "payload-processing-e2e-mt-d2d17aff" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-d2d17aff

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-b-28f011-6bcd7f4887-x7pf7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

AddedInterface

Add eth0 [10.134.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-28f011-d59dcb5b6-h9kwt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-d2d17aff-676cd69d4f

SuccessfulCreate

Created pod: payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

openshift-ingress

kubelet

payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

Created

Created container: payload-processing

openshift-ingress

kubelet

e2e-inf-b-28f011-openshift-default-65dbc584cb-fgr9q

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-3da4be

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-3da4be

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

deployment-controller

e2e-shared-a-w3-3da4be-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-3da4be-openshift-default-74cd97987d from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d

SuccessfulCreate

Created pod: e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

openshift-ingress

multus

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.593s (2.593s including waiting). Image size: 181318953 bytes.
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-3da4be: serviceaccount "payload-processing-e2e-shared-a-w3-3da4be" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-3da4be

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-3da4be: serviceaccount "payload-processing-e2e-shared-a-w3-3da4be" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-3da4be

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

AddedInterface

Add eth0 [10.134.0.82/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

ProbeError

Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Unhealthy

Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-shared-b-w3-b48b7c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b

SuccessfulCreate

Created pod: e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

openshift-ingress

multus

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-b48b7c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-b48b7c
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Started

Started container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-b48b7c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-b48b7c-77f6777557 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-b48b7c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-b48b7c: serviceaccount "payload-processing-e2e-shared-b-w3-b48b7c" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-b48b7c: serviceaccount "payload-processing-e2e-shared-b-w3-b48b7c" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

ProbeError

Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

Unhealthy

Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-d2d17aff

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-d2d17aff: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-d2d17aff": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-d2d17aff, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: d83f049d-e8ff-4bff-ac34-1864f527a274, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-d2d17aff-79db8d6c9d-gcsx5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-d2d17aff-676cd69d4f-sbcrh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-d2d17aff-openshift-default-74687dc448-c6drq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-api-a-29d3c7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-29d3c7-openshift-default-77c5dd5fb8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-a-29d3c7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-29d3c7

openshift-ingress

multus

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8

SuccessfulCreate

Created pod: e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-29d3c7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-29d3c7-5d5bc47d7f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-29d3c7: serviceaccount "payload-processing-e2e-api-a-29d3c7" not found

openshift-ingress

multus

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-29d3c7: serviceaccount "payload-processing-e2e-api-a-29d3c7" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-29d3c7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

ProbeError

Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Unhealthy

Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-api-b-0829c1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-0829c1

openshift-ingress

multus

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

AddedInterface

Add eth0 [10.134.0.95/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-b-0829c1-openshift-default-7bd67bbb64

SuccessfulCreate

Created pod: e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

openshift-ingress

deployment-controller

e2e-api-b-0829c1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-0829c1-openshift-default-7bd67bbb64 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-0829c1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-0829c1-5cf68c645f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-0829c1-5cf68c645f

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-0829c1-5cf68c645f

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-0829c1-5cf68c645f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0829c1: serviceaccount "payload-processing-e2e-api-b-0829c1" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0829c1: serviceaccount "payload-processing-e2e-api-b-0829c1" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-0829c1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-0829c1-fdf65b55d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-080801-84db66fd65-6kw77

Killing

Stopping container payload-processing

openshift-ingress

multus

payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

AddedInterface

Add eth0 [10.134.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-080801-6df746bf4-lxfmr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-080801-openshift-default-84855b7986-dx695

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-29d3c7-6bb79dbd74-8486f

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-29d3c7-5d5bc47d7f-2bnfs

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-a-29d3c7-openshift-default-77c5dd5fb8-k9dcd

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-b-0829c1

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-b-0829c1: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-b-0829c1": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-b-0829c1, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4e6bc9f1-961f-4f49-aa75-7184e0333fab, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-api-b-0829c1-5cf68c645f-nqk5z

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0829c1-fdf65b55d-9sbf7

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

Started

Started container istio-proxy
(x3)

openshift-ingress

kubelet

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

ProbeError

Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

Unhealthy

Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused

openshift-ingress

kubelet

e2e-api-b-0829c1-openshift-default-7bd67bbb64-2vlgz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-4009da-6bcf4c54b5-gfhtj

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-a-w2-4009da

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4009da: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-w2-4009da": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4009da, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4730a3f5-ce04-4eb9-aa88-4e511c6a3bc9, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-4009da-7665d67c74-cstkq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w2-4009da-openshift-default-76fc699c86-sl9b2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-60bf73-ddfdf7bbc-rsrsv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-60bf73-6b4f8f6ff4-jxmqq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-60bf73-openshift-default-66966687fb-zz7cc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-8b50f2-77b8c4548f-9drjt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-8b50f2-868dc945f8-qvlb7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-8b50f2-openshift-default-869f86786-g5ldb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-c2db32

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c2db32

openshift-ingress

deployment-controller

e2e-ipp-a-c2db32-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-c2db32-openshift-default-5d9f8564d6 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6

SuccessfulCreate

Created pod: e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

openshift-ingress

multus

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

AddedInterface

Add eth0 [10.134.0.106/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds

AddedInterface

Add eth0 [10.134.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

AddedInterface

Add eth0 [10.134.0.108/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c2db32: serviceaccount "payload-processing-e2e-ipp-a-c2db32" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c2db32: serviceaccount "payload-processing-e2e-ipp-a-c2db32" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-c2db32

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-c2db32-7cd94cf49c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-c2db32

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Unhealthy

Readiness probe failed: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

ProbeError

Readiness probe error: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

ProbeError

Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Unhealthy

Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ipp-b-5943ae-openshift-default-6cf8449d54

SuccessfulCreate

Created pod: e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-5943ae

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-5943ae

openshift-ingress

deployment-controller

e2e-ipp-b-5943ae-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-5943ae-openshift-default-6cf8449d54 from 0 to 1

openshift-ingress

multus

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

AddedInterface

Add eth0 [10.134.0.109/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-5943ae-5b54b9859c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-5943ae-5b54b9859c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5943ae: serviceaccount "payload-processing-e2e-ipp-b-5943ae" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-5943ae-5b54b9859c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-5943ae

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-5943ae-5b54b9859c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

AddedInterface

Add eth0 [10.134.0.110/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

AddedInterface

Add eth0 [10.134.0.111/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5943ae: serviceaccount "payload-processing-e2e-ipp-b-5943ae" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-5943ae

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Created

Created container: istio-proxy
(x4)

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Unhealthy

Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

ProbeError

Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-8fb478

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-8fb478

openshift-ingress

deployment-controller

e2e-ipp-cleanup-8fb478-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b

SuccessfulCreate

Created pod: e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

openshift-ingress

multus

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

AddedInterface

Add eth0 [10.133.0.86/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-8fb478

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-8fb478-6865d49749 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-8fb478

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-8fb478: serviceaccount "payload-processing-e2e-ipp-cleanup-8fb478" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk

AddedInterface

Add eth0 [10.134.0.113/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk

Started

Started container payload-pre-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-8fb478: serviceaccount "payload-processing-e2e-ipp-cleanup-8fb478" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

Unhealthy

Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

ProbeError

Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-8fb478-6865d49749-7fq7k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-8fb478-7896f9c97d-q99mk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-8fb478-openshift-default-85f99fbf4b-kmz6c

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-c2db32-6b7d868774-zzwj8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-c2db32-7cd94cf49c-p5hds

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ipp-a-c2db32-openshift-default-5d9f8564d6-mp5h6

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-5943ae-fc5f65f66-l4st2

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-5943ae-5b54b9859c-dlqwn

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ipp-b-5943ae-openshift-default-6cf8449d54-vj6cx

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-3da4be-6db8d59d69-fgqmj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-3da4be-77bcb9f4f8-6tgz7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-3da4be-openshift-default-74cd97987d-vxxp5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-b48b7c-77f6777557-27dvp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-b48b7c-599955fc9d-r7tf4

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w3-b48b7c

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b48b7c: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w3-b48b7c": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w3-b48b7c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 87791307-8818-4cfe-90f0-467ef25b1663, UID in object meta:

openshift-ingress

kubelet

e2e-shared-b-w3-b48b7c-openshift-default-644d8c64b-d8f5j

Killing

Stopping container istio-proxy