Time Namespace Component RelatedObject Reason Message

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt to ip-10-0-134-254.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-rdjfk to ip-10-0-134-254.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-r59mq

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r59mq to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

Scheduled

Successfully assigned openshift-ingress/e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

Scheduled

Successfully assigned openshift-ingress/e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-835ec8-openshift-default-6d9c668588-xht45 to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

Scheduled

Successfully assigned openshift-ingress/e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb to ip-10-0-134-254.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-27zlr

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-27zlr to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

Scheduled

Successfully assigned openshift-ingress/e2e-noten-37953c-openshift-default-f575b9d4-k7tw9 to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-r7jp6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-r7jp6 to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z to ip-10-0-134-254.ec2.internal

openshift-ingress

router-default-69bbc96588-pkg66

Scheduled

Successfully assigned openshift-ingress/router-default-69bbc96588-pkg66 to ip-10-0-140-71.ec2.internal

openshift-ingress

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498 to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2 to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8 to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

Scheduled

Successfully assigned openshift-ingress/e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

Scheduled

Successfully assigned openshift-ingress/e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w to ip-10-0-134-254.ec2.internal

openshift-ingress

kube-auth-proxy-55f6c77cbf-64gz4

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-55f6c77cbf-64gz4 to ip-10-0-140-71.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-ltqcv

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-ltqcv to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd to ip-10-0-134-254.ec2.internal

openshift-ingress

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

Scheduled

Successfully assigned openshift-ingress/e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb to ip-10-0-138-251.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-autores-7c3429-8475798fc-2k76r

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-7c3429-8475798fc-2k76r to ip-10-0-138-251.ec2.internal

openshift-ingress

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w to ip-10-0-138-251.ec2.internal

openshift-ingress

kube-auth-proxy-55f6c77cbf-cxxmq

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-55f6c77cbf-cxxmq to ip-10-0-134-254.ec2.internal

openshift-ingress

payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk to ip-10-0-138-251.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-69bbc96588 from 0 to 1
(x13)

openshift-ingress

replicaset-controller

router-default-69bbc96588

FailedCreate

Error creating: pods "router-default-69bbc96588-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x17)

openshift-ingress

replicaset-controller

router-default-69bbc96588

FailedCreate

Error creating: pods "router-default-69bbc96588-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-69bbc96588

SuccessfulCreate

Created pod: router-default-69bbc96588-pkg66
(x6)

openshift-ingress

kubelet

router-default-69bbc96588-pkg66

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-69bbc96588-pkg66

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-69bbc96588-pkg66

Created

Created container: router

openshift-ingress

kubelet

router-default-69bbc96588-pkg66

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

kubelet

router-default-69bbc96588-pkg66

Started

Started container router

openshift-ingress

multus

router-default-69bbc96588-pkg66

AddedInterface

Add eth0 [10.133.0.12/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-55f6c77cbf

FailedCreate

Error creating: pods "kube-auth-proxy-55f6c77cbf-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-55f6c77cbf from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-55f6c77cbf

SuccessfulCreate

Created pod: kube-auth-proxy-55f6c77cbf-cxxmq

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-cxxmq

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-64gz4

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-55f6c77cbf

SuccessfulCreate

Created pod: kube-auth-proxy-55f6c77cbf-64gz4

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-r59mq

AddedInterface

Add eth0 [10.132.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

kube-auth-proxy-55f6c77cbf-cxxmq

AddedInterface

Add eth0 [10.134.0.17/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-64gz4

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-55f6c77cbf-64gz4

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-cxxmq

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-r59mq

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-cxxmq

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

Created

Created container: discovery

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-64gz4

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.931s (2.931s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-64gz4

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-cxxmq

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-64gz4

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.528s (2.528s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

kube-auth-proxy-55f6c77cbf-cxxmq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.921s (2.921s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

AddedInterface

Add eth0 [10.134.0.18/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-kmtc5

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 3.244s (3.244s including waiting). Image size: 181318953 bytes.

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-27zlr

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-27zlr

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-27zlr

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-27zlr

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-27zlr

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-27zlr

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.382s (2.382s including waiting). Image size: 181126104 bytes.

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-r59mq

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r59mq

Killing

Stopping container discovery

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-rdjfk

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-r7jp6

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-ltqcv
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

multus

payload-processing-5b54b7fbb7-ltqcv

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-r7jp6

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ltqcv

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-r7jp6

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ltqcv

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.421s (2.421s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-r7jp6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.813s (2.813s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-r7jp6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-r7jp6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ltqcv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-ltqcv

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

Unhealthy

Startup probe failed: Get "http://10.134.0.27:15021/healthz/ready": dial tcp 10.134.0.27:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-rdjfk

ProbeError

Startup probe error: Get "http://10.134.0.27:15021/healthz/ready": dial tcp 10.134.0.27:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-7a4d5c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-7a4d5c

openshift-ingress

replicaset-controller

e2e-inf-a-9a49af-openshift-default-76d456f4cd

SuccessfulCreate

Created pod: e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

openshift-ingress

deployment-controller

e2e-inf-a-9a49af-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-9a49af-openshift-default-76d456f4cd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-a-9a49af

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9a49af

openshift-ingress

replicaset-controller

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6

SuccessfulCreate

Created pod: e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-0e8ffe

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-0e8ffe

openshift-ingress

deployment-controller

e2e-worker-w4-7a4d5c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-7a4d5c-openshift-default-f975bb484 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-7a4d5c-openshift-default-f975bb484

SuccessfulCreate

Created pod: e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

openshift-ingress

deployment-controller

e2e-worker-w0-0e8ffe-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6 from 0 to 1

openshift-ingress

multus

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.784s (2.784s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-7a4d5c: serviceaccount "payload-processing-e2e-worker-w4-7a4d5c" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-7a4d5c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-7a4d5c: serviceaccount "payload-processing-e2e-worker-w4-7a4d5c" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-7a4d5c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-0e8ffe

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-0e8ffe: serviceaccount "payload-processing-e2e-worker-w0-0e8ffe" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-0e8ffe

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-0e8ffe: serviceaccount "payload-processing-e2e-worker-w0-0e8ffe" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

openshift-ingress

replicaset-controller

e2e-autores-7c3429-openshift-default-54b4bc4bb6

SuccessfulCreate

Created pod: e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-9a49af-7cb84c6757

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-9a49af-7cb84c6757-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9a49af: serviceaccount "payload-processing-e2e-inf-a-9a49af" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-9a49af

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-9a49af

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-9a49af-7cb84c6757 from 0 to 1

openshift-ingress

deployment-controller

e2e-autores-7c3429-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-7c3429-openshift-default-54b4bc4bb6 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-autores-7c3429

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-7c3429

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9a49af: serviceaccount "payload-processing-e2e-inf-a-9a49af" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

openshift-ingress

kubelet

payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

Started

Started container payload-processing

openshift-ingress

multus

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-9a49af-7cb84c6757

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

openshift-ingress

multus

payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ait-1ce21877-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-1ce21877-openshift-default-cf7d965b4 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-1ce21877-openshift-default-cf7d965b4

SuccessfulCreate

Created pod: e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

openshift-ingress

gateway_labeler_controller

e2e-ait-1ce21877

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-1ce21877

openshift-ingress

multus

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-1ce21877-649bdfcd9f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-1ce21877-649bdfcd9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1ce21877: serviceaccount "payload-processing-e2e-ait-1ce21877" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-1ce21877: serviceaccount "payload-processing-e2e-ait-1ce21877" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-1ce21877

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-1ce21877-7747f49bbb from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-1ce21877

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-1ce21877-649bdfcd9f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-1ce21877-649bdfcd9f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

openshift-ingress

kubelet

payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-autores-7c3429

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-7c3429-8475798fc from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-7c3429-8475798fc

FailedCreate

Error creating: pods "payload-processing-e2e-autores-7c3429-8475798fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-7c3429: serviceaccount "payload-processing-e2e-autores-7c3429" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-7c3429-76f68fd846

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-7c3429-76f68fd846

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-7c3429-76f68fd846-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-7c3429: serviceaccount "payload-processing-e2e-autores-7c3429" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-7c3429

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-7c3429-76f68fd846 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-7c3429-8475798fc

SuccessfulCreate

Created pod: payload-processing-e2e-autores-7c3429-8475798fc-2k76r

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-autores-7c3429-8475798fc-2k76r

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-7c3429-8475798fc-2k76r

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-7c3429-8475798fc-2k76r

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-7c3429-8475798fc-2k76r

Created

Created container: payload-processing

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

ProbeError

Readiness probe error: Get "http://10.134.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Unhealthy

Readiness probe failed: Get "http://10.134.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

e2e-inf-b-4bef4d-openshift-default-64c768b77d

SuccessfulCreate

Created pod: e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k
(x2)

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Started

Started container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-inf-b-4bef4d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4bef4d
(x2)

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

deployment-controller

e2e-inf-b-4bef4d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-4bef4d-openshift-default-64c768b77d from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Created

Created container: istio-proxy

openshift-ingress

multus

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Unhealthy

Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

ProbeError

Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Unhealthy

Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

ProbeError

Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-4bef4d-6dd8c86645

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-4bef4d-6dd8c86645-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4bef4d: serviceaccount "payload-processing-e2e-inf-b-4bef4d" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-4bef4d-6dd8c86645

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-4bef4d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-4bef4d-6dd8c86645 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4bef4d: serviceaccount "payload-processing-e2e-inf-b-4bef4d" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-4bef4d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

Created

Created container: istio-proxy
(x6)

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Unhealthy

Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

Unhealthy

Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

ProbeError

Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

ProbeError

Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

ProbeError

Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

Unhealthy

Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-1ce21877-7747f49bbb-l6bqd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-1ce21877-649bdfcd9f-ljrfk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-1ce21877-openshift-default-cf7d965b4-48fw5

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-migrate-b6f4a170-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-b6f4a170-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-b6f4a170-gw

openshift-ingress

replicaset-controller

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4

SuccessfulCreate

Created pod: e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h
(x2)

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-b6f4a170-644cd88d94

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-b6f4a170

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-b6f4a170

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-b6f4a170-644cd88d94 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-b6f4a170-644cd88d94

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-b6f4a170-644cd88d94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-b6f4a170: serviceaccount "payload-processing-e2e-migrate-b6f4a170" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-b6f4a170: serviceaccount "payload-processing-e2e-migrate-b6f4a170" not found

openshift-ingress

multus

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr

Created

Created container: payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Unhealthy

Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

ProbeError

Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-migrate-b6f4a170-gw-openshift-default-ddf4b9bb4-d4h9h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-b6f4a170-5cbf7569fd-k2zbr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-b6f4a170-644cd88d94-49ttt

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-migrate-b6f4a170

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-migrate-b6f4a170: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-migrate-b6f4a170": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-migrate-b6f4a170, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4fe7cb04-6fdd-40dc-937b-58fdb23d86b0, UID in object meta:

openshift-ingress

multus

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ait-f6101af5-openshift-default-fc79c786d

SuccessfulCreate

Created pod: e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

openshift-ingress

deployment-controller

e2e-ait-f6101af5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-f6101af5-openshift-default-fc79c786d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-f6101af5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f6101af5

openshift-ingress

multus

payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-f6101af5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-f6101af5-59bcdb547d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f6101af5: serviceaccount "payload-processing-e2e-ait-f6101af5" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-f6101af5-6494486cc9

FailedCreate

Error creating: pods "payload-processing-e2e-ait-f6101af5-6494486cc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f6101af5: serviceaccount "payload-processing-e2e-ait-f6101af5" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-f6101af5-6494486cc9

SuccessfulCreate

Created pod: payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

openshift-ingress

deployment-controller

payload-processing-e2e-ait-f6101af5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-f6101af5-6494486cc9 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

ProbeError

Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

Unhealthy

Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-f6101af5-59bcdb547d-ggt5r

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-f6101af5-6494486cc9-9jlkh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-f6101af5-openshift-default-fc79c786d-lxl2c

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-derive-d6af8fa6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-d6af8fa6-openshift-default-5c784979b8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-d6af8fa6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-d6af8fa6

openshift-ingress

replicaset-controller

e2e-derive-d6af8fa6-openshift-default-5c784979b8

SuccessfulCreate

Created pod: e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

openshift-ingress

multus

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-d6af8fa6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-d6af8fa6: serviceaccount "payload-processing-e2e-derive-d6af8fa6" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-d6af8fa6-779455574f

FailedCreate

Error creating: pods "payload-processing-e2e-derive-d6af8fa6-779455574f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-d6af8fa6: serviceaccount "payload-processing-e2e-derive-d6af8fa6" not found

openshift-ingress

deployment-controller

payload-processing-e2e-derive-d6af8fa6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-d6af8fa6-779455574f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-d6af8fa6-779455574f

SuccessfulCreate

Created pod: payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

openshift-ingress

kubelet

payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-d6af8fa6-7f869ff94-8nb5c

Killing

Stopping container payload-pre-processing
(x5)

openshift-ingress

kubelet

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

ProbeError

Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

Unhealthy

Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-derive-d6af8fa6-779455574f-ltkww

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-derive-d6af8fa6-openshift-default-5c784979b8-sj5qb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-autores-7c3429-8475798fc-2k76r

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-7c3429-76f68fd846-45s9p

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-0e8ffe-7fb6ff6d7-npsrh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-0e8ffe-f55fb445d-c9hxv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-0e8ffe-openshift-default-7d749bd7f6-r7x7s

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-autores-7c3429-openshift-default-54b4bc4bb6-rrpnw

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-explicit-4915f8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-4915f8

openshift-ingress

multus

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-explicit-4915f8-openshift-default-7f9c997876

SuccessfulCreate

Created pod: e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

openshift-ingress

deployment-controller

e2e-explicit-4915f8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-4915f8-openshift-default-7f9c997876 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-4915f8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-4915f8-87fcf949d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-4915f8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-4915f8-59d587c745 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-4915f8-59d587c745

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-4915f8-59d587c745-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-4915f8: serviceaccount "payload-processing-e2e-explicit-4915f8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-4915f8-59d587c745

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-4915f8-87fcf949d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-4915f8-87fcf949d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-4915f8: serviceaccount "payload-processing-e2e-explicit-4915f8" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-4915f8-87fcf949d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

openshift-ingress

kubelet

payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

Started

Started container payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-mt-f2bc0450

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-f2bc0450

openshift-ingress

replicaset-controller

e2e-mt-f2bc0450-openshift-default-5845c6bf5b

SuccessfulCreate

Created pod: e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

openshift-ingress

deployment-controller

e2e-mt-f2bc0450-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-f2bc0450-openshift-default-5845c6bf5b from 0 to 1

openshift-ingress

multus

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

ProbeError

Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

Unhealthy

Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f2bc0450: serviceaccount "payload-processing-e2e-mt-f2bc0450" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-f2bc0450

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-f2bc0450-69bbbdc77

FailedCreate

Error creating: pods "payload-processing-e2e-mt-f2bc0450-69bbbdc77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-f2bc0450: serviceaccount "payload-processing-e2e-mt-f2bc0450" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-f2bc0450-69bbbdc77

SuccessfulCreate

Created pod: payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

openshift-ingress

deployment-controller

payload-processing-e2e-mt-f2bc0450

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-f2bc0450-69bbbdc77 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

openshift-ingress

multus

payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

ProbeError

Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

Unhealthy

Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-f2bc0450-676bc9d9f4-lb9qz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-f2bc0450-69bbbdc77-88gsn

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-f2bc0450

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-f2bc0450: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-f2bc0450": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-f2bc0450, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 499a7a0c-6143-4e2b-89dd-06dc33e4e0e7, UID in object meta:

openshift-ingress

kubelet

e2e-mt-f2bc0450-openshift-default-5845c6bf5b-fq2lk

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-shared-a-w2-65d8af-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-65d8af

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-65d8af

openshift-ingress

replicaset-controller

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d

SuccessfulCreate

Created pod: e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

openshift-ingress

multus

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4915f8-87fcf949d-gb4fp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-4915f8-59d587c745-pw4ld

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-explicit-4915f8

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-explicit-4915f8: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-explicit-4915f8": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-explicit-4915f8, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 576ca39d-f0e8-4496-b7e0-cdb57ea65e5b, UID in object meta:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-65d8af

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-65d8af: serviceaccount "payload-processing-e2e-shared-a-w2-65d8af" not found

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-65d8af

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-65d8af-7b4585756f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-65d8af: serviceaccount "payload-processing-e2e-shared-a-w2-65d8af" not found

openshift-ingress

kubelet

e2e-explicit-4915f8-openshift-default-7f9c997876-fxxk2

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-noten-37953c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-37953c-openshift-default-f575b9d4 from 0 to 1

openshift-ingress

kubelet

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

Started

Started container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-noten-37953c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-37953c

openshift-ingress

replicaset-controller

e2e-noten-37953c-openshift-default-f575b9d4

SuccessfulCreate

Created pod: e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

ProbeError

Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Unhealthy

Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-f9133e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-f9133e

openshift-ingress

deployment-controller

e2e-shared-b-w2-f9133e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-f9133e-openshift-default-68949f88f8 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8

SuccessfulCreate

Created pod: e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-9a49af-75f44ddf58-q2hkn

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-9a49af-7cb84c6757-cdftt

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-9a49af

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-9a49af: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-9a49af": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-9a49af, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8a2a33d3-87ec-4f23-85d7-4dbaff402a3c, UID in object meta:

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f9133e: serviceaccount "payload-processing-e2e-shared-b-w2-f9133e" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-f9133e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-f9133e: serviceaccount "payload-processing-e2e-shared-b-w2-f9133e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-f9133e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-f9133e-b5cf86585 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f

Started

Started container payload-processing
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Unhealthy

Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

ProbeError

Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-inf-a-9a49af-openshift-default-76d456f4cd-h2cjn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-inf-b-4bef4d-6dd8c86645-jm498

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-4bef4d-dcd47bd78-5kjc5

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb

SuccessfulCreate

Created pod: e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-72eee5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-72eee5

openshift-ingress

deployment-controller

e2e-shared-a-w3-72eee5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb from 0 to 1

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

multus

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-4bef4d-openshift-default-64c768b77d-75c6k

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-72eee5: serviceaccount "payload-processing-e2e-shared-a-w3-72eee5" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-72eee5: serviceaccount "payload-processing-e2e-shared-a-w3-72eee5" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-72eee5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-72eee5-dcd99494f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-72eee5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-b1650816

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-b1650816

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

Started

Started container payload-processing

openshift-ingress

replicaset-controller

e2e-mt-b1650816-openshift-default-6c7d5f96fb

SuccessfulCreate

Created pod: e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

openshift-ingress

deployment-controller

e2e-mt-b1650816-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-b1650816-openshift-default-6c7d5f96fb from 0 to 1

openshift-ingress

multus

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-b1650816-5cbbd65f9c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

openshift-ingress

deployment-controller

payload-processing-e2e-mt-b1650816

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-b1650816-5cbbd65f9c from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-b1650816-5cbbd65f9c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-b1650816-5cbbd65f9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-b1650816: serviceaccount "payload-processing-e2e-mt-b1650816" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-b1650816

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-b1650816-b6855766 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-b1650816-b6855766

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-b1650816-b6855766

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-b1650816-b6855766-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-b1650816: serviceaccount "payload-processing-e2e-mt-b1650816" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

Started

Started container istio-proxy

openshift-ingress

deployment-controller

e2e-shared-b-w3-3be0c5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4

SuccessfulCreate

Created pod: e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-3be0c5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-3be0c5

openshift-ingress

multus

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-3be0c5: serviceaccount "payload-processing-e2e-shared-b-w3-3be0c5" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-3be0c5: serviceaccount "payload-processing-e2e-shared-b-w3-3be0c5" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-3be0c5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

openshift-ingress

kubelet

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-37953c-openshift-default-f575b9d4-k7tw9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-3be0c5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

Unhealthy

Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

ProbeError

Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Unhealthy

Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": read tcp 10.132.0.2:43636->10.132.0.69:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

ProbeError

Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": read tcp 10.132.0.2:43636->10.132.0.69:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Unhealthy

Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

ProbeError

Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused body:

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-mt-b1650816

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-mt-b1650816: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-mt-b1650816": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-mt-b1650816, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 4fc186a0-4b6c-4515-ac00-a248bbdcc0b2, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-b1650816-b6855766-2bgsz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-b1650816-5cbbd65f9c-dv6vv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-b1650816-openshift-default-6c7d5f96fb-4h8tl

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-api-a-835ec8-openshift-default-6d9c668588

SuccessfulCreate

Created pod: e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

openshift-ingress

gateway_labeler_controller

e2e-api-a-835ec8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-835ec8

openshift-ingress

deployment-controller

e2e-api-a-835ec8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-835ec8-openshift-default-6d9c668588 from 0 to 1

openshift-ingress

multus

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-835ec8-6748b6d886

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-835ec8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-835ec8-9c864b447 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-835ec8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-835ec8-6748b6d886 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-835ec8-6748b6d886

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-835ec8-6748b6d886-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-835ec8: serviceaccount "payload-processing-e2e-api-a-835ec8" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-835ec8-9c864b447

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-835ec8-9c864b447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-835ec8: serviceaccount "payload-processing-e2e-api-a-835ec8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-835ec8-9c864b447

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Unhealthy

Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

ProbeError

Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-api-b-5c41db-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-5c41db-openshift-default-6db6b75f76 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-api-b-5c41db

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-5c41db

openshift-ingress

replicaset-controller

e2e-api-b-5c41db-openshift-default-6db6b75f76

SuccessfulCreate

Created pod: e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

openshift-ingress

multus

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-5c41db-7b77df66c9

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-5c41db-7b77df66c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-5c41db: serviceaccount "payload-processing-e2e-api-b-5c41db" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-5c41db-7b77df66c9

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-5c41db

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-5c41db-7b77df66c9 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-5c41db: serviceaccount "payload-processing-e2e-api-b-5c41db" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-5c41db

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-5c41db-bcd78b75d from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Unhealthy

Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

ProbeError

Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Created

Created container: istio-proxy
(x6)

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-7a4d5c-7b54999db-828tz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-7a4d5c-5dd74ddd9f-tnrs8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-835ec8-6748b6d886-7vmtd

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-a-835ec8

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-a-835ec8: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-a-835ec8": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-a-835ec8, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: faedb1d7-c8b9-49dd-b8c9-562a0deaa91f, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-api-a-835ec8-9c864b447-ml6qq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w4-7a4d5c-openshift-default-f975bb484-zt62w

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-a-835ec8-openshift-default-6d9c668588-xht45

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-5c41db-bcd78b75d-xnzmf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-5c41db-7b77df66c9-qgb4p

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-b-5c41db

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-b-5c41db: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-b-5c41db": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-b-5c41db, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 1f1aa3f8-7c94-4592-b32a-4a131ccc7084, UID in object meta:

openshift-ingress

kubelet

e2e-api-b-5c41db-openshift-default-6db6b75f76-r5s6h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-65d8af-7b4585756f-mz7qp

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-65d8af-7d4c796df8-cb6nl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-a-w2-65d8af-openshift-default-5cfc6c9d5d-9gd5l

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-f9133e-b5cf86585-gvs7f

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-f9133e-85d944bf74-7hqw6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-f9133e-openshift-default-68949f88f8-tb95z

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd

SuccessfulCreate

Created pod: e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

openshift-ingress

deployment-controller

e2e-ipp-a-3ba9e7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-3ba9e7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-3ba9e7

openshift-ingress

multus

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3ba9e7: serviceaccount "payload-processing-e2e-ipp-a-3ba9e7" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-3ba9e7

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-3ba9e7-55b97856f5 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3ba9e7: serviceaccount "payload-processing-e2e-ipp-a-3ba9e7" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-3ba9e7

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb

AddedInterface

Add eth0 [10.134.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Unhealthy

Readiness probe failed: Get "http://10.132.0.81:15021/healthz/ready": read tcp 10.132.0.2:43758->10.132.0.81:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

ProbeError

Readiness probe error: Get "http://10.132.0.81:15021/healthz/ready": read tcp 10.132.0.2:43758->10.132.0.81:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Unhealthy

Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

ProbeError

Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused body:

openshift-ingress

multus

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-43ef83

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-43ef83

openshift-ingress

deployment-controller

e2e-ipp-b-43ef83-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b

SuccessfulCreate

Created pod: e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-43ef83

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m

Started

Started container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-43ef83: serviceaccount "payload-processing-e2e-ipp-b-43ef83" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-43ef83-7df64c9b5

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-43ef83-7df64c9b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-43ef83: serviceaccount "payload-processing-e2e-ipp-b-43ef83" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-43ef83-7df64c9b5

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-43ef83

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-43ef83-7df64c9b5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m
(x2)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

ProbeError

Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Unhealthy

Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b

SuccessfulCreate

Created pod: e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

openshift-ingress

deployment-controller

e2e-ipp-cleanup-6b0cae-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-6b0cae-openshift-default-556df948b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-6b0cae

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-6b0cae

openshift-ingress

GatewayAuthBootstrap

e2e-ipp-cleanup-6b0cae

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ipp-cleanup-6b0cae-authn-ssl: envoyfilters.networking.istio.io "e2e-ipp-cleanup-6b0cae-authn-ssl" already exists

openshift-ingress

multus

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

AddedInterface

Add eth0 [10.132.0.83/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

Created

Created container: payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-6b0cae

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-6b0cae

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-6b0cae: serviceaccount "payload-processing-e2e-ipp-cleanup-6b0cae" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-6b0cae: serviceaccount "payload-processing-e2e-ipp-cleanup-6b0cae" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

ProbeError

Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

Unhealthy

Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-6b0cae-8665c5565c-p6mxw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-6b0cae-64d78cd4f4-jddrx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-6b0cae-openshift-default-556df948b-l6jnp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-3ba9e7-fc8cddcd-lvbpm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-3ba9e7-55b97856f5-dt8cb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-3ba9e7-openshift-default-5c8d6cbdbd-k484j

Killing

Stopping container istio-proxy

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-b-43ef83

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-43ef83: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-43ef83": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-43ef83, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ea6ec506-1fa3-4a24-8a59-60ac34ffd4bc, UID in object meta:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-43ef83-644d8bff54-zz46m

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-43ef83-7df64c9b5-m94n9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

ProbeError

Readiness probe error: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-b-43ef83-openshift-default-dcfcd4c4b-vf62w

Unhealthy

Readiness probe failed: Get "http://10.132.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-72eee5-dcd99494f-jvn7z

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-72eee5-5c7c49f4bb-z9lj7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-72eee5-openshift-default-5dbdc4d6bb-w86pw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-3be0c5-67b8d58454-tn4m6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-3be0c5-5bf9bf969-jr22k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-b-w3-3be0c5-openshift-default-6db58895f4-g267j

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503