Time Namespace Component RelatedObject Reason Message

openshift-ingress

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-92c557-openshift-default-886956d48-9825n to ip-10-0-139-118.ec2.internal

openshift-ingress

istiod-openshift-gateway-5677ccb49b-zr6b6

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-5677ccb49b-zr6b6 to ip-10-0-133-204.ec2.internal

openshift-ingress

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Scheduled

Successfully assigned openshift-ingress/e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh to ip-10-0-129-187.ec2.internal

openshift-ingress

kube-auth-proxy-58d69b594d-jqh6r

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-58d69b594d-jqh6r to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn to ip-10-0-129-187.ec2.internal

openshift-ingress

kube-auth-proxy-58d69b594d-z46js

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-58d69b594d-z46js to ip-10-0-133-204.ec2.internal

openshift-ingress

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

Scheduled

Successfully assigned openshift-ingress/e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

Scheduled

Successfully assigned openshift-ingress/e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-20ba8c-openshift-default-68c89854-kx588 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-5cc6cd444b-jsgww

Scheduled

Successfully assigned openshift-ingress/payload-processing-5cc6cd444b-jsgww to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7 to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-1b2859-84f6998448-cppr5 to ip-10-0-139-118.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-8b8fn to ip-10-0-139-118.ec2.internal

openshift-ingress

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2 to ip-10-0-139-118.ec2.internal

openshift-ingress

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Scheduled

Successfully assigned openshift-ingress/e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-687dbb6cf6-kncv4

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-687dbb6cf6-kncv4 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p to ip-10-0-139-118.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5 to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm to ip-10-0-129-187.ec2.internal

openshift-ingress

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Scheduled

Successfully assigned openshift-ingress/e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9 to ip-10-0-129-187.ec2.internal

openshift-ingress

router-default-95c8f4d49-5ncng

Scheduled

Successfully assigned openshift-ingress/router-default-95c8f4d49-5ncng to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-mt-890677f3-787d64964b-894m7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-890677f3-787d64964b-894m7 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9 to ip-10-0-139-118.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9 to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc to ip-10-0-129-187.ec2.internal

openshift-ingress

payload-processing-e2e-derive-ac8148f2-b544c7946-flv84

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-ac8148f2-b544c7946-flv84 to ip-10-0-139-118.ec2.internal
(x5)

openshift-ingress

replicaset-controller

router-default-95c8f4d49

FailedCreate

Error creating: pods "router-default-95c8f4d49-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-95c8f4d49 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer
(x17)

openshift-ingress

replicaset-controller

router-default-95c8f4d49

FailedCreate

Error creating: pods "router-default-95c8f4d49-" is forbidden: autoscaling.openshift.io/ManagementCPUsOverride the cluster does not have any nodes
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts

openshift-ingress

replicaset-controller

router-default-95c8f4d49

SuccessfulCreate

Created pod: router-default-95c8f4d49-5ncng
(x5)

openshift-ingress

kubelet

router-default-95c8f4d49-5ncng

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x6)

openshift-ingress

kubelet

router-default-95c8f4d49-5ncng

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt

openshift-ingress

kubelet

router-default-95c8f4d49-5ncng

Started

Started container router

openshift-ingress

kubelet

router-default-95c8f4d49-5ncng

Created

Created container: router

openshift-ingress

kubelet

router-default-95c8f4d49-5ncng

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:924ddd9185690e32284b189b3d0c7f005f10029b713c963227fee4da98829559" already present on machine

openshift-ingress

multus

router-default-95c8f4d49-5ncng

AddedInterface

Add eth0 [10.132.0.16/23] from ovn-kubernetes

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-58d69b594d from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-58d69b594d

SuccessfulCreate

Created pod: kube-auth-proxy-58d69b594d-jqh6r

openshift-ingress

replicaset-controller

kube-auth-proxy-58d69b594d

SuccessfulCreate

Created pod: kube-auth-proxy-58d69b594d-z46js
(x3)

openshift-ingress

replicaset-controller

kube-auth-proxy-58d69b594d

FailedCreate

Error creating: pods "kube-auth-proxy-58d69b594d-" is forbidden: error looking up service account openshift-ingress/kube-auth-proxy: serviceaccount "kube-auth-proxy" not found

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-z46js

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-jqh6r

FailedMount

MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found

openshift-ingress

multus

kube-auth-proxy-58d69b594d-z46js

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-jqh6r

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-5677ccb49b from 0 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-5677ccb49b

SuccessfulCreate

Created pod: istiod-openshift-gateway-5677ccb49b-zr6b6

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-z46js

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2"

openshift-ingress

multus

kube-auth-proxy-58d69b594d-jqh6r

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

openshift-ingress

multus

istiod-openshift-gateway-5677ccb49b-zr6b6

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-zr6b6

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31"

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-jqh6r

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-jqh6r

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-jqh6r

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 2.71s (2.71s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-z46js

Created

Created container: kube-auth-proxy

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-z46js

Started

Started container kube-auth-proxy

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-zr6b6

Created

Created container: discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-zr6b6

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:2a25d47b4bb3bf346563a0ccea986c0ab0466709ca4cb9d2666ba6a02a8a5f31" in 3.144s (3.144s including waiting). Image size: 181126104 bytes.

openshift-ingress

kubelet

kube-auth-proxy-58d69b594d-z46js

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy:v3.5.0-ea.2" in 4.476s (4.476s including waiting). Image size: 265120132 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-zr6b6

Started

Started container discovery

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-zr6b6

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-5677ccb49b-zr6b6

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-68cf4d799c

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-68cf4d799c from 0 to 1

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

Started

Started container istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.599s (2.599s including waiting). Image size: 181318953 bytes.

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

Created

Created container: istio-proxy

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

Unhealthy

Startup probe failed: Get "http://10.134.0.28:15021/healthz/ready": dial tcp 10.134.0.28:15021: connect: connection refused

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68cf4d799cq2p4r

ProbeError

Startup probe error: Get "http://10.134.0.28:15021/healthz/ready": dial tcp 10.134.0.28:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-5ccb968c66 from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-5ccb968c66

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-5ccb968c66-8b8fn

openshift-ingress

replicaset-controller

payload-processing-5cc6cd444b

SuccessfulCreate

Created pod: payload-processing-5cc6cd444b-jsgww

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5cc6cd444b from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-687dbb6cf6

FailedCreate

Error creating: pods "payload-pre-processing-687dbb6cf6-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-kncv4

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-687dbb6cf6 from 0 to 1

openshift-ingress

multus

payload-processing-5cc6cd444b-jsgww

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-5cc6cd444b

FailedCreate

Error creating: pods "payload-processing-5cc6cd444b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

replicaset-controller

payload-pre-processing-687dbb6cf6

SuccessfulCreate

Created pod: payload-pre-processing-687dbb6cf6-kncv4

openshift-ingress

multus

payload-pre-processing-687dbb6cf6-kncv4

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5cc6cd444b-jsgww

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-kncv4

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-kncv4

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-687dbb6cf6-kncv4

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.379s (2.379s including waiting). Image size: 165907129 bytes.

openshift-ingress

kubelet

payload-processing-5cc6cd444b-jsgww

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-5cc6cd444b-jsgww

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.5s (3.501s including waiting). Image size: 165907129 bytes.

openshift-ingress

kubelet

payload-processing-5cc6cd444b-jsgww

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

ProbeError

Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-5ccb968c66-8b8fn

Unhealthy

Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-ait-ecc0dac1-openshift-default-7965f995bd

SuccessfulCreate

Created pod: e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

openshift-ingress

gateway_labeler_controller

e2e-ait-ecc0dac1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ecc0dac1

openshift-ingress

multus

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ait-ecc0dac1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-ecc0dac1-openshift-default-7965f995bd from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-ecc0dac1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ecc0dac1: serviceaccount "payload-processing-e2e-ait-ecc0dac1" not found

openshift-ingress

multus

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-ecc0dac1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-ecc0dac1-655b9cbbd from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ecc0dac1-655b9cbbd

SuccessfulCreate

Created pod: payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-ecc0dac1-655b9cbbd

FailedCreate

Error creating: pods "payload-processing-e2e-ait-ecc0dac1-655b9cbbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ecc0dac1: serviceaccount "payload-processing-e2e-ait-ecc0dac1" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Unhealthy

Readiness probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

ProbeError

Readiness probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Unhealthy

Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

ProbeError

Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ait-ecc0dac1-655b9cbbd-25b7x

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-ecc0dac1-84564b7b88-m5hjp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-ecc0dac1-openshift-default-7965f995bd-qnp5c

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec"

openshift-ingress

multus

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-migrate-5f6a9856-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-migrate-5f6a9856-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-5f6a9856-gw

openshift-ingress

replicaset-controller

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956

SuccessfulCreate

Created pod: e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" in 2.688s (2.688s including waiting). Image size: 181318953 bytes.

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-5f6a9856

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-5f6a9856-978b56c5c

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-5f6a9856-978b56c5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-5f6a9856: serviceaccount "payload-processing-e2e-migrate-5f6a9856" not found

openshift-ingress

multus

payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-5f6a9856

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-5f6a9856-978b56c5c from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-5f6a9856: serviceaccount "payload-processing-e2e-migrate-5f6a9856" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-5f6a9856-978b56c5c

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

openshift-ingress

kubelet

payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

Started

Started container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-migrate-5f6a9856-978b56c5c-nsvdd

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-5f6a9856-5c4d6ddb48-jgvjp

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-5f6a9856-gw-openshift-default-7f4486f956-5f5qn

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-b42b78c0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-b42b78c0

openshift-ingress

replicaset-controller

e2e-ait-b42b78c0-openshift-default-554ddd5844

SuccessfulCreate

Created pod: e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

openshift-ingress

deployment-controller

e2e-ait-b42b78c0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-b42b78c0-openshift-default-554ddd5844 from 0 to 1

openshift-ingress

multus

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b42b78c0-67f6969f48

FailedCreate

Error creating: pods "payload-processing-e2e-ait-b42b78c0-67f6969f48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b42b78c0: serviceaccount "payload-processing-e2e-ait-b42b78c0" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-b42b78c0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-b42b78c0: serviceaccount "payload-processing-e2e-ait-b42b78c0" not found

openshift-ingress

multus

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-processing-e2e-ait-b42b78c0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-b42b78c0-67f6969f48 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-b42b78c0-67f6969f48

SuccessfulCreate

Created pod: payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

openshift-ingress

multus

payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k

Started

Started container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ait-b42b78c0-67f6969f48-k8l6w

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-b42b78c0-c5ff7dd4-r8p9k

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-b42b78c0-openshift-default-554ddd5844-d22bp

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd

SuccessfulCreate

Created pod: e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

openshift-ingress

deployment-controller

e2e-derive-ac8148f2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-ac8148f2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-ac8148f2

openshift-ingress

multus

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-ac8148f2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-ac8148f2-77879b4d68 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-ac8148f2-b544c7946

FailedCreate

Error creating: pods "payload-processing-e2e-derive-ac8148f2-b544c7946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-ac8148f2: serviceaccount "payload-processing-e2e-derive-ac8148f2" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-ac8148f2-b544c7946

SuccessfulCreate

Created pod: payload-processing-e2e-derive-ac8148f2-b544c7946-flv84
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-ac8148f2: serviceaccount "payload-processing-e2e-derive-ac8148f2" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

openshift-ingress

deployment-controller

payload-processing-e2e-derive-ac8148f2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-ac8148f2-b544c7946 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

AddedInterface

Add eth0 [10.132.0.61/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-ac8148f2-b544c7946-flv84

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-derive-ac8148f2-b544c7946-flv84

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-derive-ac8148f2-b544c7946-flv84

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-ac8148f2-b544c7946-flv84

Created

Created container: payload-processing
(x2)

openshift-ingress

kubelet

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

kubelet

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-derive-ac8148f2-b544c7946-flv84

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-ac8148f2-77879b4d68-qwpdl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-ac8148f2-openshift-default-6d4cbf9bdd-2kktw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-mt-ae9f1f22-openshift-default-7744576bb4

SuccessfulCreate

Created pod: e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

openshift-ingress

multus

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-mt-ae9f1f22-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-ae9f1f22-openshift-default-7744576bb4 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-mt-ae9f1f22

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-ae9f1f22
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ae9f1f22: serviceaccount "payload-processing-e2e-mt-ae9f1f22" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

openshift-ingress

deployment-controller

payload-processing-e2e-mt-ae9f1f22

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-ae9f1f22-b68f48b94 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-ae9f1f22-b68f48b94

FailedCreate

Error creating: pods "payload-processing-e2e-mt-ae9f1f22-b68f48b94-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-ae9f1f22: serviceaccount "payload-processing-e2e-mt-ae9f1f22" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-ae9f1f22

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

Started

Started container payload-processing

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-ae9f1f22-b68f48b94

SuccessfulCreate

Created pod: payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

AddedInterface

Add eth0 [10.132.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

Created

Created container: payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Started

Started container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-ae9f1f22-fc7fc5cbd-xcrck

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-ae9f1f22-b68f48b94-x68n8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-ae9f1f22-openshift-default-7744576bb4-vplf9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

e2e-mt-890677f3-openshift-default-ddcfb5984

SuccessfulCreate

Created pod: e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

openshift-ingress

gateway_labeler_controller

e2e-mt-890677f3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-890677f3

openshift-ingress

deployment-controller

e2e-mt-890677f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-890677f3-openshift-default-ddcfb5984 from 0 to 1

openshift-ingress

multus

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-890677f3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-890677f3-794b65987b from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-890677f3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-890677f3-787d64964b from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-890677f3-787d64964b

SuccessfulCreate

Created pod: payload-processing-e2e-mt-890677f3-787d64964b-894m7
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-890677f3-787d64964b

FailedCreate

Error creating: pods "payload-processing-e2e-mt-890677f3-787d64964b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-890677f3: serviceaccount "payload-processing-e2e-mt-890677f3" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-890677f3-794b65987b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-890677f3-794b65987b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-890677f3-794b65987b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-890677f3: serviceaccount "payload-processing-e2e-mt-890677f3" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm

Created

Created container: payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-890677f3-787d64964b-894m7

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-mt-890677f3-787d64964b-894m7

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-890677f3-787d64964b-894m7

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-890677f3-787d64964b-894m7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Started

Started container istio-proxy

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Unhealthy

Readiness probe failed: Get "http://10.132.0.66:15021/healthz/ready": read tcp 10.132.0.2:39290->10.132.0.66:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

ProbeError

Readiness probe error: Get "http://10.132.0.66:15021/healthz/ready": read tcp 10.132.0.2:39290->10.132.0.66:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Created

Created container: istio-proxy
(x4)

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-mt-890677f3-787d64964b-894m7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-890677f3-794b65987b-k9ndm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-mt-890677f3-openshift-default-ddcfb5984-mprsh

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-inf-a-20ba8c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-20ba8c-openshift-default-68c89854 from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-a-20ba8c-openshift-default-68c89854

SuccessfulCreate

Created pod: e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

openshift-ingress

gateway_labeler_controller

e2e-inf-a-20ba8c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-20ba8c

openshift-ingress

multus

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-20ba8c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

Started

Started container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-20ba8c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-20ba8c: serviceaccount "payload-processing-e2e-inf-a-20ba8c" not found

openshift-ingress

kubelet

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-20ba8c: serviceaccount "payload-processing-e2e-inf-a-20ba8c" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

AddedInterface

Add eth0 [10.132.0.73/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d

SuccessfulCreate

Created pod: e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

openshift-ingress

gateway_labeler_controller

e2e-inf-b-1b1e7a

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-1b1e7a

openshift-ingress

deployment-controller

e2e-inf-b-1b1e7a-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-1b1e7a-openshift-default-64957f9d5d from 0 to 1

openshift-ingress

multus

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-1b1e7a

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1b1e7a: serviceaccount "payload-processing-e2e-inf-b-1b1e7a" not found
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-1b1e7a: serviceaccount "payload-processing-e2e-inf-b-1b1e7a" not found

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-1b1e7a

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-1b1e7a-b555b7f7d from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Unhealthy

Readiness probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Created

Created container: istio-proxy

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

ProbeError

Readiness probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

ProbeError

Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Unhealthy

Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

ProbeError

Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Unhealthy

Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Created

Created container: istio-proxy
(x4)

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Unhealthy

Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

ProbeError

Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-inf-a-20ba8c-6dc48cf4c8-qw7nc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-20ba8c-f5b55757b-w6nkt

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-inf-a-20ba8c

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-inf-a-20ba8c: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-inf-a-20ba8c": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-inf-a-20ba8c, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 34be7770-6bbd-486a-8ea8-0db7b611317f, UID in object meta:

openshift-ingress

kubelet

e2e-inf-a-20ba8c-openshift-default-68c89854-kx588

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-1b1e7a-744c57fcb4-v6lb9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-1b1e7a-b555b7f7d-85gfb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-api-a-f397e0-openshift-default-76785b9f88

SuccessfulCreate

Created pod: e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

openshift-ingress

gateway_labeler_controller

e2e-api-a-f397e0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-f397e0

openshift-ingress

deployment-controller

e2e-api-a-f397e0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-f397e0-openshift-default-76785b9f88 from 0 to 1

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-b-1b1e7a-openshift-default-64957f9d5d-4mvkn

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-f397e0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-f397e0-b6bbfc748 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-f397e0-b897c75dd

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-f397e0-b897c75dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f397e0: serviceaccount "payload-processing-e2e-api-a-f397e0" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-f397e0: serviceaccount "payload-processing-e2e-api-a-f397e0" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-f397e0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-f397e0-b897c75dd from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-f397e0-b897c75dd

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

openshift-ingress

kubelet

payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

Created

Created container: payload-processing

openshift-ingress

multus

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

AddedInterface

Add eth0 [10.132.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

Started

Started container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

AddedInterface

Add eth0 [10.132.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Created

Created container: payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-api-b-1b2859

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-1b2859

openshift-ingress

deployment-controller

e2e-api-b-1b2859-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-1b2859-openshift-default-76859d9644 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-b-1b2859-openshift-default-76859d9644

SuccessfulCreate

Created pod: e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

openshift-ingress

multus

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

AddedInterface

Add eth0 [10.132.0.81/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-1b2859-84f6998448

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-1b2859-84f6998448-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1b2859: serviceaccount "payload-processing-e2e-api-b-1b2859" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-1b2859

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-1b2859-65f68d76b4 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-1b2859

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-1b2859-84f6998448 from 0 to 1
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-1b2859: serviceaccount "payload-processing-e2e-api-b-1b2859" not found

openshift-ingress

kubelet

payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-1b2859-84f6998448

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

openshift-ingress

kubelet

payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Unhealthy

Readiness probe failed: dial tcp 10.132.0.79:9004: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Unhealthy

Liveness probe failed: dial tcp 10.132.0.79:9004: connect: connection refused

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Unhealthy

Readiness probe failed: Get "http://10.132.0.81:15021/healthz/ready": read tcp 10.132.0.2:39762->10.132.0.81:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

ProbeError

Readiness probe error: Get "http://10.132.0.81:15021/healthz/ready": read tcp 10.132.0.2:39762->10.132.0.81:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x4)

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

ProbeError

Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Unhealthy

Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-api-a-f397e0-b897c75dd-z98ng

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-f397e0-b6bbfc748-f2dv6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-a-f397e0-openshift-default-76785b9f88-9g6zl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-api-b-1b2859-84f6998448-cppr5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-1b2859-65f68d76b4-9cw9n

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-1b2859-openshift-default-76859d9644-gcs2c

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

multus

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-a-1badbb-openshift-default-6448f7dfb8

SuccessfulCreate

Created pod: e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

openshift-ingress

deployment-controller

e2e-shared-a-1badbb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-1badbb-openshift-default-6448f7dfb8 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-shared-a-1badbb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-1badbb

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-1badbb: serviceaccount "payload-processing-e2e-shared-a-1badbb" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-1badbb-7ff76ff76b

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-1badbb-7ff76ff76b

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-1badbb-7ff76ff76b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-1badbb: serviceaccount "payload-processing-e2e-shared-a-1badbb" not found

openshift-ingress

multus

payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-1badbb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-1badbb-595f44df4d from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8

Started

Started container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-1badbb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-1badbb-7ff76ff76b from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc

Started

Started container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-shared-b-92c557

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-92c557

openshift-ingress

deployment-controller

e2e-shared-b-92c557-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-92c557-openshift-default-886956d48 from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-92c557-openshift-default-886956d48

SuccessfulCreate

Created pod: e2e-shared-b-92c557-openshift-default-886956d48-9825n

openshift-ingress

multus

e2e-shared-b-92c557-openshift-default-886956d48-9825n

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-92c557-6f4d7cf958

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-92c557-6f4d7cf958-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-92c557: serviceaccount "payload-processing-e2e-shared-b-92c557" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-92c557

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-92c557-6f4d7cf958 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-92c557-6f4d7cf958

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-92c557: serviceaccount "payload-processing-e2e-shared-b-92c557" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-92c557

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg

Started

Started container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg

Created

Created container: payload-processing

openshift-ingress

multus

payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg

AddedInterface

Add eth0 [10.132.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

Created

Created container: istio-proxy
(x3)

openshift-ingress

kubelet

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

ProbeError

Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

Unhealthy

Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Unhealthy

Readiness probe failed: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

ProbeError

Readiness probe error: Get "http://10.134.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Started

Started container istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x3)

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

ProbeError

Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused body:
(x3)

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Unhealthy

Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-dc1efb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-dc1efb

openshift-ingress

deployment-controller

e2e-ipp-a-dc1efb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b

SuccessfulCreate

Created pod: e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

openshift-ingress

multus

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

AddedInterface

Add eth0 [10.132.0.93/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

Started

Started container payload-processing
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-dc1efb: serviceaccount "payload-processing-e2e-ipp-a-dc1efb" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-dc1efb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-dc1efb: serviceaccount "payload-processing-e2e-ipp-a-dc1efb" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-dc1efb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

multus

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

AddedInterface

Add eth0 [10.132.0.94/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

Created

Created container: payload-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

Started

Started container payload-pre-processing

openshift-ingress

multus

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-ipp-b-9b749c-openshift-default-54b78f7676

SuccessfulCreate

Created pod: e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

openshift-ingress

deployment-controller

e2e-ipp-b-9b749c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-9b749c-openshift-default-54b78f7676 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-9b749c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-9b749c
(x2)

openshift-ingress

kubelet

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

Started

Started container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-9b749c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-9b749c-5fc6885d68 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

Started

Started container payload-pre-processing

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

AddedInterface

Add eth0 [10.132.0.95/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

Created

Created container: payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-9b749c-5fc6885d68

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-9b749c-5fc6885d68

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-9b749c-5fc6885d68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9b749c: serviceaccount "payload-processing-e2e-ipp-b-9b749c" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9b749c: serviceaccount "payload-processing-e2e-ipp-b-9b749c" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-9b749c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-9b749c-78d7987948 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9

Started

Started container payload-processing

openshift-ingress

multus

payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9

Created

Created container: payload-processing
(x4)

openshift-ingress

kubelet

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

ProbeError

Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

Unhealthy

Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Started

Started container istio-proxy
(x4)

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Unhealthy

Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

ProbeError

Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-ipp-cleanup-29b6e5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-cleanup-29b6e5

openshift-ingress

deployment-controller

e2e-ipp-cleanup-29b6e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd

SuccessfulCreate

Created pod: e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

openshift-ingress

multus

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

AddedInterface

Add eth0 [10.132.0.97/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-cleanup-29b6e5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-cleanup-29b6e5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-29b6e5: serviceaccount "payload-processing-e2e-ipp-cleanup-29b6e5" not found
(x4)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-cleanup-29b6e5: serviceaccount "payload-processing-e2e-ipp-cleanup-29b6e5" not found

openshift-ingress

multus

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq

AddedInterface

Add eth0 [10.132.0.98/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

AddedInterface

Add eth0 [10.134.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

Created

Created container: payload-pre-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

Started

Started container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq

Created

Created container: payload-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq

Started

Started container payload-processing
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

Created

Created container: istio-proxy
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:2b5f5aa5ee9974269d8e3666b1bfc58da10c172bf3f1d6defd555fbd1ac9a6ec" already present on machine
(x2)

openshift-ingress

kubelet

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

Started

Started container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ipp-cleanup-29b6e5-9b4f89c98-z7chq

Killing

Stopping container payload-processing
(x4)

openshift-ingress

kubelet

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

Unhealthy

Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

ProbeError

Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-cleanup-29b6e5-557786c876-ntrq5

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-cleanup-29b6e5-openshift-default-747c7c56cd-7gknm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-dc1efb-b88f6756d-cww5p

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-dc1efb-6bb5d85cdd-w7l8z

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-dc1efb-openshift-default-66b79c4b5b-hpfsx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-9b749c-78d7987948-4cz5k

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-9b749c-5fc6885d68-j77b9

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ipp-b-9b749c-openshift-default-54b78f7676-chxw2

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-1badbb-595f44df4d-mm2k8

Killing

Stopping container payload-pre-processing

openshift-ingress

endpoint-controller

payload-pre-processing-e2e-shared-a-1badbb

FailedToUpdateEndpoint

Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-shared-a-1badbb: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-shared-a-1badbb": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-shared-a-1badbb, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 61267084-ab27-4b3a-899c-4a3290489522, UID in object meta:

openshift-ingress

kubelet

payload-processing-e2e-shared-a-1badbb-7ff76ff76b-mklzc

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-a-1badbb-openshift-default-6448f7dfb8-lnggz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-92c557-66cf84bf6d-zjkp7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-92c557-6f4d7cf958-mxbgg

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-92c557-openshift-default-886956d48-9825n

Killing

Stopping container istio-proxy