Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-px5dq

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-px5dq to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

Scheduled

Successfully assigned openshift-ingress/e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

Scheduled

Successfully assigned openshift-ingress/e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9 to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-44e517-6f764db755-sshjh to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8 to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

Scheduled

Successfully assigned openshift-ingress/e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v to ip-10-0-135-209.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg to ip-10-0-135-209.ec2.internal

openshift-ingress

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42 to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

Scheduled

Successfully assigned openshift-ingress/e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Scheduled

Successfully assigned openshift-ingress/e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s to ip-10-0-136-118.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

Scheduled

Successfully assigned openshift-ingress/e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8 to ip-10-0-136-118.ec2.internal

openshift-ingress

router-default-5f57db6868-lgg22

Scheduled

Successfully assigned openshift-ingress/router-default-5f57db6868-lgg22 to ip-10-0-135-209.ec2.internal

openshift-ingress

router-default-5f57db6868-lgg22

FailedScheduling

0/2 nodes are available: 2 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.

openshift-ingress

payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6 to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8 to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-454lz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-454lz to ip-10-0-135-209.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-bplw6

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-bplw6 to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4 to ip-10-0-136-118.ec2.internal

openshift-ingress

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98 to ip-10-0-140-93.ec2.internal

openshift-ingress

payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-5ee44a89-8577659c69-r986q to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Scheduled

Successfully assigned openshift-ingress/e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78 to ip-10-0-136-118.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-xcdvq

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-xcdvq to ip-10-0-140-93.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-r7l2b

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-r7l2b to ip-10-0-140-93.ec2.internal

openshift-ingress

kube-auth-proxy-7f65bc968c-wszkq

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-7f65bc968c-wszkq to ip-10-0-136-118.ec2.internal

openshift-ingress

payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7 to ip-10-0-140-93.ec2.internal

openshift-ingress

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82 to ip-10-0-136-118.ec2.internal

openshift-ingress

kube-auth-proxy-7f65bc968c-vt5vn

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-7f65bc968c-vt5vn to ip-10-0-140-93.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-5f57db6868 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer
(x12)

openshift-ingress

replicaset-controller

router-default-5f57db6868

FailedCreate

Error creating: pods "router-default-5f57db6868-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x16)

openshift-ingress

replicaset-controller

router-default-5f57db6868

FailedCreate

Error creating: pods "router-default-5f57db6868-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

replicaset-controller

router-default-5f57db6868

SuccessfulCreate

Created pod: router-default-5f57db6868-lgg22
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x6)

openshift-ingress

kubelet

router-default-5f57db6868-lgg22

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-5f57db6868-lgg22

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

multus

router-default-5f57db6868-lgg22

AddedInterface

Add eth0 [10.132.0.7/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-5f57db6868-lgg22

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

kubelet

router-default-5f57db6868-lgg22

Created

Created container: router

openshift-ingress

kubelet

router-default-5f57db6868-lgg22

Started

Started container router

openshift-ingress

replicaset-controller

kube-auth-proxy-7f65bc968c

SuccessfulCreate

Created pod: kube-auth-proxy-7f65bc968c-wszkq

openshift-ingress

replicaset-controller

kube-auth-proxy-7f65bc968c

SuccessfulCreate

Created pod: kube-auth-proxy-7f65bc968c-vt5vn

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-vt5vn

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-7f65bc968c

FailedCreate

Error creating: pods "kube-auth-proxy-7f65bc968c-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-wszkq

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-7f65bc968c from 0 to 2

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-r7l2b

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-vt5vn

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-wszkq

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

kube-auth-proxy-7f65bc968c-wszkq

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

openshift-ingress

multus

kube-auth-proxy-7f65bc968c-vt5vn

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-r7l2b

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-wszkq

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-wszkq

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-wszkq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.965s (2.965s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.441s (3.441s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

Created

Created container: discovery

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-vt5vn

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-vt5vn

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-7f65bc968c-vt5vn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 4.107s (4.108s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.837s (2.837s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-xsdpd

Started

Started container istio-proxy

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-xcdvq

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-xcdvq

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-xcdvq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" already present on machine

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-xcdvq

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-xcdvq

Created

Created container: discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-r7l2b

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-r7l2b
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-bplw6

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-bplw6

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-454lz

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-px5dq
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-454lz

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-454lz

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-px5dq

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-5b54b7fbb7-px5dq

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-px5dq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.412s (1.412s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-px5dq

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-px5dq

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-454lz

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.856s (2.856s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-454lz

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-454lz

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-bplw6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-bplw6

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-bplw6

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-bplw6

ProbeError

Startup probe error: Get "http://10.133.0.26:15021/healthz/ready": dial tcp 10.133.0.26:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-bplw6

Unhealthy

Startup probe failed: Get "http://10.133.0.26:15021/healthz/ready": dial tcp 10.133.0.26:15021: connect: connection refused

openshift-ingress

multus

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w5-e047d8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-e047d8

openshift-ingress

deployment-controller

e2e-worker-w5-e047d8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w5-e047d8-openshift-default-557df7456b from 0 to 1

openshift-ingress

multus

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-a-e5568d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-e5568d-openshift-default-589747df4d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-35f5fd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-35f5fd

openshift-ingress

gateway_labeler_controller

e2e-inf-a-e5568d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-e5568d

openshift-ingress

replicaset-controller

e2e-inf-a-e5568d-openshift-default-589747df4d

SuccessfulCreate

Created pod: e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

openshift-ingress

deployment-controller

e2e-worker-w0-35f5fd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb

SuccessfulCreate

Created pod: e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

openshift-ingress

GatewayAuthBootstrap

e2e-inf-a-e5568d

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-inf-a-e5568d-authn-ssl: envoyfilters.networking.istio.io "e2e-inf-a-e5568d-authn-ssl" already exists

openshift-ingress

replicaset-controller

e2e-worker-w5-e047d8-openshift-default-557df7456b

SuccessfulCreate

Created pod: e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

openshift-ingress

multus

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e5568d: serviceaccount "payload-processing-e2e-inf-a-e5568d" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-e5568d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-e5568d-566c97bb8c

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-e5568d-566c97bb8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e5568d: serviceaccount "payload-processing-e2e-inf-a-e5568d" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-e5568d-566c97bb8c

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-e5568d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-e5568d-566c97bb8c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-35f5fd: serviceaccount "payload-processing-e2e-worker-w0-35f5fd" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-35f5fd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-35f5fd-d979cbb8f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-35f5fd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-35f5fd: serviceaccount "payload-processing-e2e-worker-w0-35f5fd" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

openshift-ingress

multus

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

Created

Created container: payload-pre-processing
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w5-e047d8-7d788748df

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w5-e047d8-7d788748df-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-e047d8: serviceaccount "payload-processing-e2e-worker-w5-e047d8" not found

openshift-ingress

multus

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w5-e047d8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-e047d8: serviceaccount "payload-processing-e2e-worker-w5-e047d8" not found

openshift-ingress

deployment-controller

e2e-autores-ddbefa-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-ddbefa-openshift-default-5cd48889cd from 0 to 1

openshift-ingress

replicaset-controller

e2e-autores-ddbefa-openshift-default-5cd48889cd

SuccessfulCreate

Created pod: e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

openshift-ingress

gateway_labeler_controller

e2e-autores-ddbefa

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-ddbefa

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w5-e047d8-7d788748df

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w5-e047d8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w5-e047d8-7d788748df from 0 to 1

openshift-ingress

multus

payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

Created

Created container: payload-processing

openshift-ingress

deployment-controller

e2e-ait-0e078853-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-0e078853-openshift-default-7cf5d46d6d from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-0e078853-openshift-default-7cf5d46d6d

SuccessfulCreate

Created pod: e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

openshift-ingress

gateway_labeler_controller

e2e-ait-0e078853

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-0e078853

openshift-ingress

multus

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-autores-ddbefa

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-ddbefa-8448f787b5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-ddbefa

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-ddbefa-5747dc84fd from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-ddbefa-8448f787b5

SuccessfulCreate

Created pod: payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-ddbefa-8448f787b5

FailedCreate

Error creating: pods "payload-processing-e2e-autores-ddbefa-8448f787b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-ddbefa: serviceaccount "payload-processing-e2e-autores-ddbefa" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-ddbefa: serviceaccount "payload-processing-e2e-autores-ddbefa" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Unhealthy

Readiness probe failed: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

multus

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

ProbeError

Readiness probe error: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

replicaset-controller

e2e-inf-b-44e517-openshift-default-59cf864b8

SuccessfulCreate

Created pod: e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

openshift-ingress

deployment-controller

e2e-inf-b-44e517-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-44e517-openshift-default-59cf864b8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-inf-b-44e517

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-44e517

openshift-ingress

deployment-controller

payload-processing-e2e-ait-0e078853

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-0e078853-9f65f94c5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-0e078853

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-0e078853-7498845cf4 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-0e078853-9f65f94c5

FailedCreate

Error creating: pods "payload-processing-e2e-ait-0e078853-9f65f94c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0e078853: serviceaccount "payload-processing-e2e-ait-0e078853" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-0e078853-7498845cf4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-0e078853-7498845cf4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-0e078853-7498845cf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-0e078853: serviceaccount "payload-processing-e2e-ait-0e078853" not found

openshift-ingress

multus

payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-0e078853-9f65f94c5

SuccessfulCreate

Created pod: payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

Created

Created container: istio-proxy

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-44e517-6f764db755

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-44e517-666645dd94

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j
(x5)

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

ProbeError

Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused body:

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-44e517

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-44e517-6f764db755 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-44e517-666645dd94

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-44e517-666645dd94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-44e517: serviceaccount "payload-processing-e2e-inf-b-44e517" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-44e517

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-44e517-666645dd94 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-44e517-6f764db755

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-44e517-6f764db755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-44e517: serviceaccount "payload-processing-e2e-inf-b-44e517" not found
(x5)

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Unhealthy

Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

ProbeError

Startup probe error: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Unhealthy

Startup probe failed: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused

openshift-ingress

multus

payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

Started

Started container payload-processing
(x7)

openshift-ingress

kubelet

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

ProbeError

Startup probe error: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

Unhealthy

Startup probe failed: Get "http://10.133.0.33:15021/healthz/ready": dial tcp 10.133.0.33:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

ProbeError

Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

Unhealthy

Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

Unhealthy

Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

ProbeError

Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x6)

openshift-ingress

kubelet

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

ProbeError

Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

Unhealthy

Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ait-0e078853-9f65f94c5-f9g6x

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-0e078853-7498845cf4-kbz98

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-0e078853-openshift-default-7cf5d46d6d-ql22m

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-migrate-595d23dd-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-595d23dd-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-595d23dd-gw

openshift-ingress

replicaset-controller

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d

SuccessfulCreate

Created pod: e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

openshift-ingress

multus

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-595d23dd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-595d23dd-f7d5d9f4b from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-595d23dd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-595d23dd-5bf9587466 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-595d23dd: serviceaccount "payload-processing-e2e-migrate-595d23dd" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-595d23dd: serviceaccount "payload-processing-e2e-migrate-595d23dd" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-migrate-595d23dd-gw-openshift-default-f699c4b6d-bq7sq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-595d23dd-5bf9587466-fz6rd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-595d23dd-f7d5d9f4b-jp6jj

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-ait-8eafdc51

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8eafdc51

openshift-ingress

replicaset-controller

e2e-ait-8eafdc51-openshift-default-cfc745794

SuccessfulCreate

Created pod: e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

openshift-ingress

multus

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ait-8eafdc51-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-8eafdc51-openshift-default-cfc745794 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-8eafdc51

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-8eafdc51-7664789686 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ait-8eafdc51

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-8eafdc51-6f48d848f7 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8eafdc51-6f48d848f7

FailedCreate

Error creating: pods "payload-processing-e2e-ait-8eafdc51-6f48d848f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8eafdc51: serviceaccount "payload-processing-e2e-ait-8eafdc51" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8eafdc51-7664789686

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8eafdc51-7664789686

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-8eafdc51-7664789686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8eafdc51: serviceaccount "payload-processing-e2e-ait-8eafdc51" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8eafdc51-6f48d848f7

SuccessfulCreate

Created pod: payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

openshift-ingress

kubelet

payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8eafdc51-7664789686-zbv8x

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-8eafdc51-6f48d848f7-jmr6m

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-derive-a1fb6d89

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-a1fb6d89

openshift-ingress

kubelet

e2e-ait-8eafdc51-openshift-default-cfc745794-7hxd9

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4

SuccessfulCreate

Created pod: e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

openshift-ingress

deployment-controller

e2e-derive-a1fb6d89-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-a1fb6d89-openshift-default-7d797dd4b4 from 0 to 1

openshift-ingress

multus

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-a1fb6d89: serviceaccount "payload-processing-e2e-derive-a1fb6d89" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-a1fb6d89-68df956b65

FailedCreate

Error creating: pods "payload-processing-e2e-derive-a1fb6d89-68df956b65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-a1fb6d89: serviceaccount "payload-processing-e2e-derive-a1fb6d89" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-a1fb6d89-68df956b65

SuccessfulCreate

Created pod: payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

openshift-ingress

deployment-controller

payload-processing-e2e-derive-a1fb6d89

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-a1fb6d89-68df956b65 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-a1fb6d89

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-autores-ddbefa-8448f787b5-wgkv7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-ddbefa-5747dc84fd-jvd6v

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-explicit-b91829-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-b91829-openshift-default-65d8cc6bcb from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-explicit-b91829

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-b91829

openshift-ingress

replicaset-controller

e2e-explicit-b91829-openshift-default-65d8cc6bcb

SuccessfulCreate

Created pod: e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

openshift-ingress

kubelet

e2e-autores-ddbefa-openshift-default-5cd48889cd-9hv6x

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

AddedInterface

Add eth0 [10.133.0.49/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

ProbeError

Readiness probe error: Get "http://10.133.0.47:15021/healthz/ready": read tcp 10.133.0.2:34630->10.133.0.47:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Unhealthy

Readiness probe failed: Get "http://10.133.0.47:15021/healthz/ready": read tcp 10.133.0.2:34630->10.133.0.47:15021: read: connection reset by peer

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-b91829-6c5c668d97

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5
(x4)

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Unhealthy

Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-derive-a1fb6d89

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-derive-a1fb6d89: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-derive-a1fb6d89": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-derive-a1fb6d89, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: e9451442-ab17-40c9-b8f0-881d2a498166, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-b91829-6c5c668d97

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-b91829-6c5c668d97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-b91829: serviceaccount "payload-processing-e2e-explicit-b91829" not found

openshift-ingress

kubelet

payload-processing-e2e-derive-a1fb6d89-68df956b65-stkcb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-a1fb6d89-6cf98fcd7d-t7gx5

Killing

Stopping container payload-pre-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-b91829-547595bf78

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-b91829-547595bf78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-b91829: serviceaccount "payload-processing-e2e-explicit-b91829" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-b91829-547595bf78

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-b91829

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-b91829-547595bf78 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5

AddedInterface

Add eth0 [10.134.0.68/23] from ovn-kubernetes
(x4)

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

ProbeError

Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-derive-a1fb6d89-openshift-default-7d797dd4b4-hft78

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-b91829

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-b91829-6c5c668d97 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Unhealthy

Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

ProbeError

Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-b91829-547595bf78-6dbsz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-explicit-b91829-6c5c668d97-2mfv5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-noten-0a22a7-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-0a22a7-openshift-default-7848467d94 from 0 to 1

openshift-ingress

kubelet

e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

Started

Started container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-noten-0a22a7

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-0a22a7

openshift-ingress

replicaset-controller

e2e-noten-0a22a7-openshift-default-7848467d94

SuccessfulCreate

Created pod: e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-explicit-b91829-openshift-default-65d8cc6bcb-dch82

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-mt-06ac85db-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-06ac85db-openshift-default-6df8fb649b from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-06ac85db-openshift-default-6df8fb649b

SuccessfulCreate

Created pod: e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

openshift-ingress

gateway_labeler_controller

e2e-mt-06ac85db

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-06ac85db

openshift-ingress

multus

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-mt-06ac85db

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-06ac85db-76888b5b97 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-06ac85db-76888b5b97

SuccessfulCreate

Created pod: payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-06ac85db-76888b5b97

FailedCreate

Error creating: pods "payload-processing-e2e-mt-06ac85db-76888b5b97-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-06ac85db: serviceaccount "payload-processing-e2e-mt-06ac85db" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-06ac85db

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-06ac85db-85b74b7686 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-06ac85db-85b74b7686

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-06ac85db-85b74b7686

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-06ac85db-85b74b7686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-06ac85db: serviceaccount "payload-processing-e2e-mt-06ac85db" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-mt-06ac85db-76888b5b97-f8qtf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-06ac85db-85b74b7686-vvmzv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-06ac85db-openshift-default-6df8fb649b-6x7tl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-noten-0a22a7-openshift-default-7848467d94-k7qhm

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-296002

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-296002

openshift-ingress

deployment-controller

e2e-shared-a-w2-296002-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-296002-openshift-default-7674c75784 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w2-296002-openshift-default-7674c75784

SuccessfulCreate

Created pod: e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

openshift-ingress

multus

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-296002-6555c6d555

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-296002

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-296002-6555c6d555 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-296002-6555c6d555

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-296002-6555c6d555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-296002: serviceaccount "payload-processing-e2e-shared-a-w2-296002" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-296002: serviceaccount "payload-processing-e2e-shared-a-w2-296002" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-296002

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

Created

Created container: payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-e5568d

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-e5568d: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-e5568d": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-e5568d, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 7643e8dc-7601-4df9-8e81-fff99cf61481, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-inf-a-e5568d-566c97bb8c-8tfpm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-e5568d-678b8b9dbb-gbgv8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-inf-a-e5568d-openshift-default-589747df4d-88vwr

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8

SuccessfulCreate

Created pod: e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

openshift-ingress

deployment-controller

e2e-shared-b-w2-365ea1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-365ea1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-365ea1
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

multus

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-44e517-6f764db755-sshjh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-44e517-666645dd94-msl9j

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

kubelet

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

ProbeError

Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

Unhealthy

Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-365ea1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-365ea1: serviceaccount "payload-processing-e2e-shared-b-w2-365ea1" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-365ea1: serviceaccount "payload-processing-e2e-shared-b-w2-365ea1" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-365ea1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-44e517-openshift-default-59cf864b8-6ns42

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-a7df81

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-a7df81

openshift-ingress

replicaset-controller

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74

SuccessfulCreate

Created pod: e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

openshift-ingress

deployment-controller

e2e-shared-a-w3-a7df81-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74 from 0 to 1

openshift-ingress

multus

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-a7df81

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-a7df81

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-a7df81: serviceaccount "payload-processing-e2e-shared-a-w3-a7df81" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-a7df81: serviceaccount "payload-processing-e2e-shared-a-w3-a7df81" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

AddedInterface

Add eth0 [10.134.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

AddedInterface

Add eth0 [10.134.0.81/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-5ee44a89-openshift-default-868ff99448

SuccessfulCreate

Created pod: e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

openshift-ingress

deployment-controller

e2e-mt-5ee44a89-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-5ee44a89-openshift-default-868ff99448 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-5ee44a89

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-5ee44a89

openshift-ingress

multus

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-5ee44a89: serviceaccount "payload-processing-e2e-mt-5ee44a89" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

openshift-ingress

deployment-controller

payload-processing-e2e-mt-5ee44a89

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-5ee44a89-8577659c69 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-5ee44a89-8577659c69

FailedCreate

Error creating: pods "payload-processing-e2e-mt-5ee44a89-8577659c69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-5ee44a89: serviceaccount "payload-processing-e2e-mt-5ee44a89" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-5ee44a89-8577659c69

SuccessfulCreate

Created pod: payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-5ee44a89

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

Created

Created container: payload-pre-processing

openshift-ingress

GatewayAuthBootstrap

e2e-shared-b-w3-1b68db

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-shared-b-w3-1b68db-authn-ssl: envoyfilters.networking.istio.io "e2e-shared-b-w3-1b68db-authn-ssl" already exists

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-1b68db

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-1b68db

openshift-ingress

kubelet

payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

Started

Started container payload-processing

openshift-ingress

deployment-controller

e2e-shared-b-w3-1b68db-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-1b68db-openshift-default-7496b948dd from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd

SuccessfulCreate

Created pod: e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

openshift-ingress

kubelet

payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

AddedInterface

Add eth0 [10.134.0.84/23] from ovn-kubernetes

openshift-ingress

multus

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

AddedInterface

Add eth0 [10.133.0.63/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-1b68db

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-1b68db-776b8564c9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-1b68db: serviceaccount "payload-processing-e2e-shared-b-w3-1b68db" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-1b68db

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-1b68db: serviceaccount "payload-processing-e2e-shared-b-w3-1b68db" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6

AddedInterface

Add eth0 [10.134.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Unhealthy

Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

ProbeError

Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Unhealthy

Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

ProbeError

Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-5ee44a89-5f84f9dbf4-hn84k

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-5ee44a89-8577659c69-r986q

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-5ee44a89-openshift-default-868ff99448-rpc7s

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-api-a-0de41b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-0de41b-openshift-default-7f69dd45dd from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-0de41b-openshift-default-7f69dd45dd

SuccessfulCreate

Created pod: e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

openshift-ingress

gateway_labeler_controller

e2e-api-a-0de41b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-0de41b

openshift-ingress

multus

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-0de41b-b766857bb

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-0de41b-b766857bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0de41b: serviceaccount "payload-processing-e2e-api-a-0de41b" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-0de41b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-0de41b-5c964d5cd7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-0de41b-5c964d5cd7

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-0de41b-5c964d5cd7

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-0de41b-5c964d5cd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0de41b: serviceaccount "payload-processing-e2e-api-a-0de41b" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-0de41b-b766857bb

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-0de41b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-0de41b-b766857bb from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

AddedInterface

Add eth0 [10.134.0.94/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Unhealthy

Readiness probe failed: Get "http://10.133.0.72:15021/healthz/ready": read tcp 10.133.0.2:38926->10.133.0.72:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

gateway_labeler_controller

e2e-api-b-0102fa

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-0102fa

openshift-ingress

deployment-controller

e2e-api-b-0102fa-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-0102fa-openshift-default-6b8fd5fdb from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-b-0102fa-openshift-default-6b8fd5fdb

SuccessfulCreate

Created pod: e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8
(x2)

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

ProbeError

Readiness probe error: Get "http://10.133.0.72:15021/healthz/ready": read tcp 10.133.0.2:38926->10.133.0.72:15021: read: connection reset by peer body:

openshift-ingress

multus

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

AddedInterface

Add eth0 [10.133.0.74/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-0102fa

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-0102fa-77945f78f4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-0102fa

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-0102fa-6f8c658957 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-0102fa-77945f78f4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-0102fa-6f8c658957

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

openshift-ingress

multus

payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

AddedInterface

Add eth0 [10.134.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

Created

Created container: payload-processing
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-0102fa-77945f78f4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-0102fa-77945f78f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0102fa: serviceaccount "payload-processing-e2e-api-b-0102fa" not found

openshift-ingress

kubelet

payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

AddedInterface

Add eth0 [10.134.0.95/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-0102fa-6f8c658957

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-0102fa-6f8c658957-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0102fa: serviceaccount "payload-processing-e2e-api-b-0102fa" not found
(x6)

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Unhealthy

Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

ProbeError

Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-35f5fd-d979cbb8f-7lkkw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-35f5fd-757d78b8c8-hpqv8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w0-35f5fd-openshift-default-68dcb7ddb-7m69m

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-0de41b-b766857bb-p5fdf

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-api-a-0de41b

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-api-a-0de41b: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-api-a-0de41b": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-api-a-0de41b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ebbeb828-9b70-4f91-af51-83c4bc9b2445, UID in object meta:
(x2)

openshift-ingress

kubelet

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

Created

Created container: istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-api-a-0de41b-5c964d5cd7-r77bb

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

ProbeError

Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

Unhealthy

Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-a-0de41b-openshift-default-7f69dd45dd-hthqt

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-0102fa-77945f78f4-x7q5b

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-0102fa-6f8c658957-4fgf6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-api-b-0102fa-openshift-default-6b8fd5fdb-kzzj8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-296002-55b86c647d-gclqx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-296002-6555c6d555-nk4zg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w2-296002-openshift-default-7674c75784-94twp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-365ea1-574ffb5d85-4xrpt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-365ea1-58bfc7f998-b59h8

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-365ea1-openshift-default-5f8f9f85c8-phghl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w5-e047d8-59f5b5fbcd-72h6v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w5-e047d8-7d788748df-whk7v

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w5-e047d8-openshift-default-557df7456b-4qw6d

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-ipp-a-ad0222-openshift-default-778686b7d4

SuccessfulCreate

Created pod: e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

openshift-ingress

deployment-controller

e2e-ipp-a-ad0222-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-ad0222-openshift-default-778686b7d4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-ad0222

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-ad0222

openshift-ingress

multus

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

AddedInterface

Add eth0 [10.133.0.86/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-ad0222-c59474794

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-ad0222

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-ad0222-74ccc8db79 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-ad0222

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-ad0222-c59474794 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-ad0222-c59474794

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-ad0222-c59474794-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-ad0222: serviceaccount "payload-processing-e2e-ipp-a-ad0222" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-ad0222-74ccc8db79

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-ad0222-74ccc8db79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-ad0222: serviceaccount "payload-processing-e2e-ipp-a-ad0222" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-ad0222-74ccc8db79

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8

SuccessfulCreate

Created pod: e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-03eca6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-03eca6

openshift-ingress

deployment-controller

e2e-ipp-b-03eca6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-03eca6-openshift-default-6969dcf7d8 from 0 to 1

openshift-ingress

multus

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

AddedInterface

Add eth0 [10.133.0.87/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-03eca6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-03eca6: serviceaccount "payload-processing-e2e-ipp-b-03eca6" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-03eca6-6558849f7f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-03eca6-6558849f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-03eca6: serviceaccount "payload-processing-e2e-ipp-b-03eca6" not found
(x2)

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-03eca6-6558849f7f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-03eca6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-03eca6-6558849f7f from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Started

Started container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Unhealthy

Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

ProbeError

Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

ProbeError

Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Unhealthy

Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused

openshift-ingress

GatewayAuthBootstrap

e2e-ipp-cleanup-dcbfbc

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-ipp-cleanup-dcbfbc-authn-ssl: envoyfilters.networking.istio.io "e2e-ipp-cleanup-dcbfbc-authn-ssl" already exists

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-dcbfbc

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-dcbfbc

openshift-ingress

deployment-controller

e2e-ipp-cleanup-dcbfbc-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b

SuccessfulCreate

Created pod: e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

openshift-ingress

multus

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

AddedInterface

Add eth0 [10.133.0.89/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-dcbfbc: serviceaccount "payload-processing-e2e-ipp-cleanup-dcbfbc" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-dcbfbc

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-dcbfbc: serviceaccount "payload-processing-e2e-ipp-cleanup-dcbfbc" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-dcbfbc

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

AddedInterface

Add eth0 [10.134.0.102/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

AddedInterface

Add eth0 [10.134.0.103/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

Unhealthy

Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

ProbeError

Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-dcbfbc-5b785c676c-hkms2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-dcbfbc-5d949b9d66-8c9hh

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ipp-cleanup-dcbfbc

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dcbfbc: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-cleanup-dcbfbc": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-dcbfbc, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a0cd56c7-9887-44cd-a922-1e95562d5e2e, UID in object meta:

openshift-ingress

kubelet

e2e-ipp-cleanup-dcbfbc-openshift-default-7f8c88fb7b-469gf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-ad0222-74ccc8db79-7ccx2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-ad0222-c59474794-rh2bt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-a-ad0222-openshift-default-778686b7d4-h2px8

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-03eca6-6558849f7f-2lrv9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-03eca6-7b479bbb54-4kj49

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ipp-b-03eca6-openshift-default-6969dcf7d8-28f4f

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-a7df81-7d68c6ff77-52h2k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-a7df81-78cf4c7b46-c4s4s

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-a-w3-a7df81

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a7df81: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-w3-a7df81": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-w3-a7df81, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8e969294-b1bb-4e52-aa1a-210e29a9af91, UID in object meta:

openshift-ingress

kubelet

e2e-shared-a-w3-a7df81-openshift-default-7b5ccb5d74-mqs2t

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-1b68db-776b8564c9-jhscg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-1b68db-75d5b4d7f8-knpj6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w3-1b68db-openshift-default-7496b948dd-5j9m4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: