Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq to ip-10-0-129-183.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4 to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Scheduled

Successfully assigned openshift-ingress/e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8 to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69 to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7 to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt to ip-10-0-141-110.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-fkggk

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-fkggk to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq to ip-10-0-141-110.ec2.internal

openshift-ingress

istiod-openshift-gateway-7b7995d5cd-zwv8w

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7b7995d5cd-zwv8w to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88 to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht to ip-10-0-141-110.ec2.internal

openshift-ingress

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Scheduled

Successfully assigned openshift-ingress/e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt to ip-10-0-129-183.ec2.internal

openshift-ingress

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54 to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5 to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65 to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

Scheduled

Successfully assigned openshift-ingress/e2e-mt-9064d2be-openshift-default-78549465b-vnfgv to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57 to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-processing-74969cc5f8-jjnsc

Scheduled

Successfully assigned openshift-ingress/payload-processing-74969cc5f8-jjnsc to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-59fbc47bf8-kbl4z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-59fbc47bf8-kbl4z to ip-10-0-141-110.ec2.internal

openshift-ingress

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

Scheduled

Successfully assigned openshift-ingress/e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7 to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j to ip-10-0-129-183.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-cg52h to ip-10-0-141-110.ec2.internal

openshift-ingress

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp to ip-10-0-129-183.ec2.internal

openshift-ingress

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs to ip-10-0-141-110.ec2.internal

openshift-ingress

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897 to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4 to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g to ip-10-0-141-110.ec2.internal

openshift-ingress

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv to ip-10-0-129-183.ec2.internal

openshift-ingress

kube-auth-proxy-687d9c5d49-wjrp4

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-687d9c5d49-wjrp4 to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d to ip-10-0-128-189.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5 to ip-10-0-128-189.ec2.internal

openshift-ingress

router-default-5ddcf9f498-ghttm

Scheduled

Successfully assigned openshift-ingress/router-default-5ddcf9f498-ghttm to ip-10-0-128-189.ec2.internal

openshift-ingress

router-default-5ddcf9f498-ghttm

FailedScheduling

0/2 nodes are available: 2 node(s) had untolerated taint(s). no new claims to deallocate, preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.

openshift-ingress

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7 to ip-10-0-129-183.ec2.internal

openshift-ingress

kube-auth-proxy-687d9c5d49-qph89

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-687d9c5d49-qph89 to ip-10-0-128-189.ec2.internal

openshift-ingress

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

Scheduled

Successfully assigned openshift-ingress/e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz to ip-10-0-141-110.ec2.internal

openshift-ingress

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6 to ip-10-0-129-183.ec2.internal

openshift-ingress

payload-processing-5b54b7fbb7-2jrlc

Scheduled

Successfully assigned openshift-ingress/payload-processing-5b54b7fbb7-2jrlc to ip-10-0-141-110.ec2.internal

openshift-ingress

payload-pre-processing-558d9b8847-2nr6n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-558d9b8847-2nr6n to ip-10-0-141-110.ec2.internal

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-5ddcf9f498 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x11)

openshift-ingress

replicaset-controller

router-default-5ddcf9f498

FailedCreate

Error creating: pods "router-default-5ddcf9f498-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x16)

openshift-ingress

replicaset-controller

router-default-5ddcf9f498

FailedCreate

Error creating: pods "router-default-5ddcf9f498-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

replicaset-controller

router-default-5ddcf9f498

SuccessfulCreate

Created pod: router-default-5ddcf9f498-ghttm
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x6)

openshift-ingress

kubelet

router-default-5ddcf9f498-ghttm

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x6)

openshift-ingress

kubelet

router-default-5ddcf9f498-ghttm

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-5ddcf9f498-ghttm

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:48526e08a2a06d05806dbcf3ddb069e5182371667d9e9f7d2cf09f60cd247d20" already present on machine

openshift-ingress

kubelet

router-default-5ddcf9f498-ghttm

Started

Started container router

openshift-ingress

multus

router-default-5ddcf9f498-ghttm

AddedInterface

Add eth0 [10.132.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-5ddcf9f498-ghttm

Created

Created container: router
(x4)

openshift-ingress

replicaset-controller

kube-auth-proxy-687d9c5d49

FailedCreate

Error creating: pods "kube-auth-proxy-687d9c5d49-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

replicaset-controller

kube-auth-proxy-687d9c5d49

SuccessfulCreate

Created pod: kube-auth-proxy-687d9c5d49-qph89

openshift-ingress

replicaset-controller

kube-auth-proxy-687d9c5d49

SuccessfulCreate

Created pod: kube-auth-proxy-687d9c5d49-wjrp4

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-687d9c5d49 from 0 to 2

openshift-ingress

multus

kube-auth-proxy-687d9c5d49-wjrp4

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-qph89

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-wjrp4

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-687d9c5d49-qph89

AddedInterface

Add eth0 [10.132.0.22/23] from ovn-kubernetes

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7b7995d5cd from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-7b7995d5cd-zwv8w

AddedInterface

Add eth0 [10.132.0.23/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulCreate

Created pod: istiod-openshift-gateway-7b7995d5cd-zwv8w

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-qph89

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-wjrp4

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-qph89

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-qph89

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.697s (2.697s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-wjrp4

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.705s (2.705s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-687d9c5d49-wjrp4

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.384s (3.384s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-d66cd8c7c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-d66cd8c7c from 0 to 1

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-d66cd8c7c-smtj4

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 3.361s (3.361s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-fkggk

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-fkggk

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fkggk

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fkggk

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 2.313s (2.313s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fkggk

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-fkggk

Started

Started container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7b7995d5cd

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7b7995d5cd-zwv8w

openshift-ingress

kubelet

istiod-openshift-gateway-7b7995d5cd-zwv8w

Killing

Stopping container discovery

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7b7995d5cd from 1 to 0
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-78bb78ff78 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-78bb78ff78-cg52h

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-78bb78ff78

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-78bb78ff78-cg52h

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 3.197s (3.197s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5b54b7fbb7 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-59fbc47bf8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

SuccessfulCreate

Created pod: payload-pre-processing-59fbc47bf8-kbl4z
(x5)

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

FailedCreate

Error creating: pods "payload-processing-5b54b7fbb7-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-processing-5b54b7fbb7

SuccessfulCreate

Created pod: payload-processing-5b54b7fbb7-2jrlc
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-59fbc47bf8

FailedCreate

Error creating: pods "payload-pre-processing-59fbc47bf8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

multus

payload-processing-5b54b7fbb7-2jrlc

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-2jrlc

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-59fbc47bf8-kbl4z

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kbl4z

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kbl4z

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-2jrlc

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.992s (1.992s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-2jrlc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-2jrlc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kbl4z

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.993s (1.993s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kbl4z

Started

Started container payload-pre-processing

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

Unhealthy

Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-78bb78ff78-cg52h

ProbeError

Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-5b54b7fbb7-2jrlc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-59fbc47bf8-kbl4z

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2nr6n

Pulling

Pulling image "quay.io/maas/odh-praxis-extproc:pr699-76cb977"

openshift-ingress

multus

payload-pre-processing-558d9b8847-2nr6n

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes
(x3)

openshift-ingress

replicaset-controller

payload-pre-processing-558d9b8847

FailedCreate

Error creating: pods "payload-pre-processing-558d9b8847-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-processing-74969cc5f8-jjnsc

Pulling

Pulling image "quay.io/maas/odh-praxis-extproc:pr699-76cb977"

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-558d9b8847 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-processing-74969cc5f8

FailedCreate

Error creating: pods "payload-processing-74969cc5f8-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-558d9b8847

SuccessfulCreate

Created pod: payload-pre-processing-558d9b8847-2nr6n

openshift-ingress

replicaset-controller

payload-processing-74969cc5f8

SuccessfulCreate

Created pod: payload-processing-74969cc5f8-jjnsc

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-74969cc5f8 from 0 to 1

openshift-ingress

multus

payload-processing-74969cc5f8-jjnsc

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2nr6n

Pulled

Successfully pulled image "quay.io/maas/odh-praxis-extproc:pr699-76cb977" in 2.307s (2.307s including waiting). Image size: 120164829 bytes.

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2nr6n

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-jjnsc

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-558d9b8847-2nr6n

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-jjnsc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-74969cc5f8-jjnsc

Pulled

Successfully pulled image "quay.io/maas/odh-praxis-extproc:pr699-76cb977" in 2.282s (2.282s including waiting). Image size: 120164829 bytes.

openshift-ingress

replicaset-controller

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8

SuccessfulCreate

Created pod: e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

openshift-ingress

deployment-controller

e2e-inf-a-155ffe-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-155ffe-openshift-default-dd6f8f5c8 from 0 to 1

openshift-ingress

GatewayAuthBootstrap

e2e-inf-a-155ffe

ReconcileError

Failed to reconcile EnvoyFilter: failed to create EnvoyFilter: could not CREATE EnvoyFilter openshift-ingress/e2e-inf-a-155ffe-authn-ssl: envoyfilters.networking.istio.io "e2e-inf-a-155ffe-authn-ssl" already exists

openshift-ingress

gateway_labeler_controller

e2e-inf-a-155ffe

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-155ffe

openshift-ingress

multus

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-155ffe-8b846877c

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-155ffe-8b846877c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-155ffe: serviceaccount "payload-processing-e2e-inf-a-155ffe" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-155ffe: serviceaccount "payload-processing-e2e-inf-a-155ffe" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-155ffe-8b846877c

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-155ffe

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-155ffe-7d466d8674 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-155ffe

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-155ffe-8b846877c from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

AddedInterface

Add eth0 [10.134.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

e2e-ait-ee96ca62-openshift-default-c4d995fd

SuccessfulCreate

Created pod: e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

openshift-ingress

deployment-controller

e2e-ait-ee96ca62-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-ee96ca62-openshift-default-c4d995fd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-ee96ca62

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ee96ca62

openshift-ingress

multus

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ee96ca62-64874d4bfc

SuccessfulCreate

Created pod: payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ee96ca62-64874d4bfc

FailedCreate

Error creating: pods "payload-processing-e2e-ait-ee96ca62-64874d4bfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ee96ca62: serviceaccount "payload-processing-e2e-ait-ee96ca62" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-ee96ca62

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-ee96ca62-64874d4bfc from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ee96ca62-857464575d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-ee96ca62-857464575d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ee96ca62: serviceaccount "payload-processing-e2e-ait-ee96ca62" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ee96ca62-857464575d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-ee96ca62

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-ee96ca62-857464575d from 0 to 1

openshift-ingress

multus

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.276s (2.276s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.306s (2.306s including waiting). Image size: 166449312 bytes.

openshift-ingress

kubelet

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

ProbeError

Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": read tcp 10.134.0.2:44718->10.134.0.52:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Unhealthy

Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": read tcp 10.134.0.2:44718->10.134.0.52:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-inf-b-1b147a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-1b147a

openshift-ingress

multus

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-b-1b147a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-1b147a-openshift-default-546b9c4784 from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-b-1b147a-openshift-default-546b9c4784

SuccessfulCreate

Created pod: e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1b147a: serviceaccount "payload-processing-e2e-inf-b-1b147a" not found

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-1b147a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-1b147a-56d66fb6b7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-1b147a-56d66fb6b7

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

openshift-ingress

multus

payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-1b147a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-1b147a-56d66fb6b7

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-1b147a-56d66fb6b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1b147a: serviceaccount "payload-processing-e2e-inf-b-1b147a" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ee96ca62-857464575d-v7t65

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ee96ca62-64874d4bfc-tdsjt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-migrate-c353d682-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-c353d682-gw-openshift-default-654d459956 from 0 to 1

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ait-ee96ca62-openshift-default-c4d995fd-2blzt

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

gateway_labeler_controller

e2e-migrate-c353d682-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-c353d682-gw

openshift-ingress

replicaset-controller

e2e-migrate-c353d682-gw-openshift-default-654d459956

SuccessfulCreate

Created pod: e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

openshift-ingress

multus

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-c353d682

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-c353d682-7bbcfc6686 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-c353d682-7bbcfc6686

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-c353d682-7bbcfc6686-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-c353d682: serviceaccount "payload-processing-e2e-migrate-c353d682" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-c353d682: serviceaccount "payload-processing-e2e-migrate-c353d682" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-c353d682-7bbcfc6686

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-c353d682

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-c353d682-556fb99bf4 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

AddedInterface

Add eth0 [10.134.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

Created

Created container: payload-processing
(x5)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

ProbeError

Readiness probe error: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Unhealthy

Readiness probe failed: Get "http://10.133.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

ProbeError

Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Unhealthy

Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-migrate-c353d682-7bbcfc6686-xsj69

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-c353d682-556fb99bf4-6nl9w

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-c353d682-gw-openshift-default-654d459956-vjgvk

Killing

Stopping container istio-proxy

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-migrate-c353d682

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-migrate-c353d682: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-migrate-c353d682": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-migrate-c353d682, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: a0c920d1-dcda-4b76-a1ac-eb63ccdd85f4, UID in object meta:

openshift-ingress

replicaset-controller

e2e-ait-b02eba29-openshift-default-7ffcb64954

SuccessfulCreate

Created pod: e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

openshift-ingress

deployment-controller

e2e-ait-b02eba29-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-b02eba29-openshift-default-7ffcb64954 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-b02eba29

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-b02eba29

openshift-ingress

multus

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b02eba29-b5475f798

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-b02eba29-b5475f798-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b02eba29: serviceaccount "payload-processing-e2e-ait-b02eba29" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-b02eba29

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-b02eba29-745659bb49 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b02eba29-b5475f798

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b02eba29-745659bb49

SuccessfulCreate

Created pod: payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-b02eba29

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-b02eba29-b5475f798 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b02eba29-745659bb49

FailedCreate

Error creating: pods "payload-processing-e2e-ait-b02eba29-745659bb49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b02eba29: serviceaccount "payload-processing-e2e-ait-b02eba29" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Unhealthy

Readiness probe failed: Get "http://10.133.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

ProbeError

Readiness probe error: Get "http://10.133.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b02eba29-b5475f798-wxpfq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-b02eba29-745659bb49-tvwfn

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-ait-b02eba29

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ait-b02eba29: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ait-b02eba29": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ait-b02eba29, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6a659e70-4877-4efa-9c0a-c2c0bd3f078b, UID in object meta:

openshift-ingress

deployment-controller

e2e-derive-f6336aa2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-f6336aa2-openshift-default-69d966677b from 0 to 1

openshift-ingress

replicaset-controller

e2e-derive-f6336aa2-openshift-default-69d966677b

SuccessfulCreate

Created pod: e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

openshift-ingress

kubelet

e2e-ait-b02eba29-openshift-default-7ffcb64954-bvff8

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-derive-f6336aa2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-f6336aa2

openshift-ingress

multus

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-f6336aa2-5787744f5c

FailedCreate

Error creating: pods "payload-processing-e2e-derive-f6336aa2-5787744f5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-f6336aa2: serviceaccount "payload-processing-e2e-derive-f6336aa2" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-f6336aa2-5787744f5c

SuccessfulCreate

Created pod: payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

openshift-ingress

deployment-controller

payload-processing-e2e-derive-f6336aa2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-f6336aa2-5787744f5c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-f6336aa2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-f6336aa2-555489fff4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-f6336aa2-555489fff4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-f6336aa2-555489fff4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-f6336aa2-555489fff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-f6336aa2: serviceaccount "payload-processing-e2e-derive-f6336aa2" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

ProbeError

Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

Unhealthy

Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused

openshift-ingress

kubelet

e2e-derive-f6336aa2-openshift-default-69d966677b-6tf6c

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-derive-f6336aa2-5787744f5c-sxbrz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-f6336aa2-555489fff4-lvjxr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-155ffe-7d466d8674-xldmf

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-155ffe-8b846877c-f69zf

Killing

Stopping container payload-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-155ffe

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-155ffe: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-155ffe": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-155ffe, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 69c4bac4-8f7b-44da-bfd3-5479ad197fd5, UID in object meta:

openshift-ingress

kubelet

e2e-inf-a-155ffe-openshift-default-dd6f8f5c8-wm74g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b147a-6dd579bc98-s8dzt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b147a-56d66fb6b7-hrdht

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-42b26e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-42b26e

openshift-ingress

replicaset-controller

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6

SuccessfulCreate

Created pod: e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

openshift-ingress

multus

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-a-w2-42b26e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-42b26e: serviceaccount "payload-processing-e2e-shared-a-w2-42b26e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-42b26e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

ProbeError

Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-42b26e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b147a-openshift-default-546b9c4784-hnbtk

Unhealthy

Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-42b26e: serviceaccount "payload-processing-e2e-shared-a-w2-42b26e" not found

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd

Started

Started container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Unhealthy

Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

ProbeError

Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Unhealthy

Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

ProbeError

Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-7b7b3c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-7b7b3c

openshift-ingress

replicaset-controller

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8

SuccessfulCreate

Created pod: e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

openshift-ingress

multus

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-shared-b-w2-7b7b3c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-7b7b3c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-7b7b3c: serviceaccount "payload-processing-e2e-shared-b-w2-7b7b3c" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-7b7b3c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-7b7b3c: serviceaccount "payload-processing-e2e-shared-b-w2-7b7b3c" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Unhealthy

Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": read tcp 10.133.0.2:43724->10.133.0.52:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

ProbeError

Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": read tcp 10.133.0.2:43724->10.133.0.52:15021: read: connection reset by peer body:

openshift-ingress

deployment-controller

e2e-mt-9064d2be-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-9064d2be-openshift-default-78549465b from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-9064d2be

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-9064d2be

openshift-ingress

replicaset-controller

e2e-mt-9064d2be-openshift-default-78549465b

SuccessfulCreate

Created pod: e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

openshift-ingress

multus

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Unhealthy

Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

ProbeError

Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

Started

Started container payload-processing

openshift-ingress

deployment-controller

payload-processing-e2e-mt-9064d2be

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-9064d2be-cd7b4c7d9 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9064d2be-6f7485c98

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-9064d2be-6f7485c98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9064d2be: serviceaccount "payload-processing-e2e-mt-9064d2be" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-9064d2be-6f7485c98

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-9064d2be

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-9064d2be-6f7485c98 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9064d2be-cd7b4c7d9

SuccessfulCreate

Created pod: payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

openshift-ingress

kubelet

payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

Created

Created container: payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-9064d2be-cd7b4c7d9

FailedCreate

Error creating: pods "payload-processing-e2e-mt-9064d2be-cd7b4c7d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-9064d2be: serviceaccount "payload-processing-e2e-mt-9064d2be" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

Unhealthy

Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

ProbeError

Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-mt-9064d2be-cd7b4c7d9-v4s88

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-9064d2be-6f7485c98-l7lgl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-9064d2be-openshift-default-78549465b-vnfgv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d

SuccessfulCreate

Created pod: e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

openshift-ingress

deployment-controller

e2e-shared-a-w1-7f9f55-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w1-7f9f55-openshift-default-8975b688d from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w1-7f9f55

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-7f9f55

openshift-ingress

multus

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w1-7f9f55

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w1-7f9f55

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w1-7f9f55-669854568c from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-7f9f55: serviceaccount "payload-processing-e2e-shared-a-w1-7f9f55" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-7f9f55-669854568c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w1-7f9f55-669854568c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w1-7f9f55-669854568c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-7f9f55: serviceaccount "payload-processing-e2e-shared-a-w1-7f9f55" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w

Started

Started container payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

Created

Created container: istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w1-d2de76

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-d2de76

openshift-ingress

deployment-controller

e2e-shared-b-w1-d2de76-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b

SuccessfulCreate

Created pod: e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

openshift-ingress

multus

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-d2de76: serviceaccount "payload-processing-e2e-shared-b-w1-d2de76" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-d2de76: serviceaccount "payload-processing-e2e-shared-b-w1-d2de76" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w1-d2de76

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w1-d2de76

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

Unhealthy

Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

ProbeError

Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused body:

openshift-ingress

multus

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-mt-cc429a8e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-cc429a8e

openshift-ingress

deployment-controller

e2e-mt-cc429a8e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-cc429a8e-openshift-default-6f4c7db78d from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-cc429a8e-openshift-default-6f4c7db78d

SuccessfulCreate

Created pod: e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-cc429a8e: serviceaccount "payload-processing-e2e-mt-cc429a8e" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5

SuccessfulCreate

Created pod: payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

openshift-ingress

deployment-controller

payload-processing-e2e-mt-cc429a8e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-cc429a8e-5bf49cc7b5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-cc429a8e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5

FailedCreate

Error creating: pods "payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-cc429a8e: serviceaccount "payload-processing-e2e-mt-cc429a8e" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

Started

Started container istio-proxy
(x7)

openshift-ingress

kubelet

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

ProbeError

Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

Unhealthy

Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-mt-cc429a8e-5bf49cc7b5-5nfqw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-cc429a8e-5c57f5b8b-7x7qp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-cc429a8e-openshift-default-6f4c7db78d-shfv7

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-api-a-a1650b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-a1650b

openshift-ingress

deployment-controller

e2e-api-a-a1650b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-a1650b-openshift-default-7f6dcbc8bb from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb

SuccessfulCreate

Created pod: e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

openshift-ingress

multus

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

AddedInterface

Add eth0 [10.133.0.61/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-a1650b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-a1650b-7949db9c6f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-a1650b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a1650b: serviceaccount "payload-processing-e2e-api-a-a1650b" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7

AddedInterface

Add eth0 [10.134.0.72/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-a1650b-7949db9c6f

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-a1650b-7949db9c6f

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-a1650b-7949db9c6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a1650b: serviceaccount "payload-processing-e2e-api-a-a1650b" not found

openshift-ingress

kubelet

payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x5)

openshift-ingress

kubelet

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

Unhealthy

Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

ProbeError

Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-api-b-6b0cb9

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-6b0cb9

openshift-ingress

deployment-controller

e2e-api-b-6b0cb9-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-6b0cb9-openshift-default-6b774f69d6 from 0 to 1

openshift-ingress

multus

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-b-6b0cb9-openshift-default-6b774f69d6

SuccessfulCreate

Created pod: e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

openshift-ingress

multus

payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-6b0cb9-88bc7c597

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-6b0cb9-88bc7c597-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-6b0cb9: serviceaccount "payload-processing-e2e-api-b-6b0cb9" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-6b0cb9-88bc7c597

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-6b0cb9: serviceaccount "payload-processing-e2e-api-b-6b0cb9" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-6b0cb9

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-6b0cb9-88bc7c597 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-6b0cb9

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk
(x2)

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

ProbeError

Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Unhealthy

Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-a1650b-85bd5fd5fd-c42b7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-a1650b-7949db9c6f-xszjs

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-030981

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-030981

openshift-ingress

deployment-controller

e2e-ipp-a-030981-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-030981-openshift-default-786fdb6448 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-030981-openshift-default-786fdb6448

SuccessfulCreate

Created pod: e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

openshift-ingress

multus

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-api-a-a1650b-openshift-default-7f6dcbc8bb-tt8lp

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-030981

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-030981-5b9b75845f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-030981-5b9b75845f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-030981

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-030981: serviceaccount "payload-processing-e2e-ipp-a-030981" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-030981-5b9b75845f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-030981-5b9b75845f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-030981: serviceaccount "payload-processing-e2e-ipp-a-030981" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-6b0cb9-5758b8cf7d-5zmtk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-6b0cb9-88bc7c597-xwc9d

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Started

Started container istio-proxy

openshift-ingress

replicaset-controller

e2e-ipp-b-0249d8-openshift-default-646fd965c4

SuccessfulCreate

Created pod: e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

openshift-ingress

deployment-controller

e2e-ipp-b-0249d8-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-0249d8-openshift-default-646fd965c4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-0249d8

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-0249d8

openshift-ingress

multus

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

AddedInterface

Add eth0 [10.133.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-b-6b0cb9-openshift-default-6b774f69d6-r7mwv

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-0249d8-597dcfbc85

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-0249d8-597dcfbc85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0249d8: serviceaccount "payload-processing-e2e-ipp-b-0249d8" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-0249d8

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-0249d8: serviceaccount "payload-processing-e2e-ipp-b-0249d8" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-0249d8-597dcfbc85

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-0249d8

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-0249d8-597dcfbc85 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w1-7f9f55-57894ddd6b-ghz2v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w1-7f9f55-669854568c-zlx7w

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w1-7f9f55-openshift-default-8975b688d-vnrb6

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

ProbeError

Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

Unhealthy

Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w1-d2de76-85bd49cd9-hbck5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w1-d2de76-58c96699d-cjdbn

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-b-w1-d2de76

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2de76: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-b-w1-d2de76": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-b-w1-d2de76, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f5190a6f-0281-47d0-826b-ed041eb3c724, UID in object meta:

openshift-ingress

kubelet

e2e-shared-b-w1-d2de76-openshift-default-5f8c5d784b-ql5vq

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ipp-cleanup-c04390-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-c04390-openshift-default-74db84fd86 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-c04390

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-c04390

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86

SuccessfulCreate

Created pod: e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

openshift-ingress

multus

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-c04390

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-c04390

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-c04390-6585674cb9 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-c04390: serviceaccount "payload-processing-e2e-ipp-cleanup-c04390" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-c04390: serviceaccount "payload-processing-e2e-ipp-cleanup-c04390" not found

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

ProbeError

Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": read tcp 10.133.0.2:45726->10.133.0.71:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Unhealthy

Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": read tcp 10.133.0.2:45726->10.133.0.71:15021: read: connection reset by peer
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

ProbeError

Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-c04390-6585674cb9-gsdnz

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-c04390-56d6c8b7c4-jhbkp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-c04390-openshift-default-74db84fd86-kxkql

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-030981-7845f8fdfd-48ptq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-030981-5b9b75845f-f8f7f

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-a-030981-openshift-default-786fdb6448-hfdg7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-0249d8-7b55b995f8-9cv57

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-0249d8-597dcfbc85-k5twl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-0249d8-openshift-default-646fd965c4-x7z54

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-42b26e-5f9b444654-s4bw4

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-42b26e-5bfbb77c9-bkzjd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-w2-42b26e-openshift-default-55cd788bc6-k5897

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-7b7b3c-77c7d8dc79-9cfxx

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-7b7b3c-688574886-bbrr5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w2-7b7b3c-openshift-default-6d44877fb8-5t42j

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: