Time Namespace Component RelatedObject Reason Message

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-r7bct to ip-10-0-139-211.ec2.internal

openshift-ingress

kube-auth-proxy-74f99fd9b4-jrw8c

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-74f99fd9b4-jrw8c to ip-10-0-141-154.ec2.internal

openshift-ingress

kube-auth-proxy-74f99fd9b4-854np

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-74f99fd9b4-854np to ip-10-0-139-211.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-9mzhr to ip-10-0-139-154.ec2.internal

kuadrant-system

limitador-limitador-7c9b87d574-zznzj

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-zznzj to ip-10-0-139-154.ec2.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-vtfw2

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-vtfw2 to ip-10-0-139-154.ec2.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8bzhx to ip-10-0-139-211.ec2.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw to ip-10-0-139-211.ec2.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-nnx7p

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-nnx7p to ip-10-0-141-154.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-8lk25

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-8lk25 to ip-10-0-139-154.ec2.internal

cert-manager

cert-manager-597b95cb84-67rqd

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-67rqd to ip-10-0-139-211.ec2.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-46lnp

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-46lnp to ip-10-0-139-154.ec2.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-56bs5

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-56bs5 to ip-10-0-139-211.ec2.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-5jfpf

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-5jfpf to ip-10-0-141-154.ec2.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-r99sj

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-r99sj to ip-10-0-139-154.ec2.internal

openshift-jobset-operator

jobset-controller-manager-67b799d75-xbgnr

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-67b799d75-xbgnr to ip-10-0-139-211.ec2.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-b42wp

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-b42wp to ip-10-0-139-211.ec2.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-dspzv

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-dspzv to ip-10-0-141-154.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6 to ip-10-0-139-154.ec2.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-c8z66

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-c8z66 to ip-10-0-141-154.ec2.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-4jntq

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-4jntq to ip-10-0-139-154.ec2.internal

kuadrant-system

limitador-limitador-74dfcfb894-vn568

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-vn568 to ip-10-0-141-154.ec2.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx to ip-10-0-139-211.ec2.internal

openshift-ingress

router-default-55bb6c544c-dhlks

Scheduled

Successfully assigned openshift-ingress/router-default-55bb6c544c-dhlks to ip-10-0-139-154.ec2.internal

openshift-ingress

router-default-55bb6c544c-dhlks

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-55bb6c544c-dhlks

FailedScheduling

no nodes available to schedule pods

llm

echo-simulator-kserve-64b6f8f96c-qpzbq

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-qpzbq to ip-10-0-139-154.ec2.internal

openshift-ingress

payload-processing-55fbbbf9dc-bhvqh

Scheduled

Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-bhvqh to ip-10-0-139-154.ec2.internal

openshift-ingress

payload-pre-processing-7b98bb678c-cmbvc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-cmbvc to ip-10-0-139-211.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-6d9f996676-xtvml

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-6d9f996676-xtvml to ip-10-0-139-211.ec2.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv to ip-10-0-139-211.ec2.internal

kuadrant-system

authorino-56c6c9ddd5-l7nkf

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-l7nkf to ip-10-0-141-154.ec2.internal

kuadrant-system

authorino-operator-7b7bb4bc77-n5j5c

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-n5j5c to ip-10-0-139-154.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-6d9f996676-whcsd

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-6d9f996676-whcsd to ip-10-0-141-154.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5 to ip-10-0-139-154.ec2.internal

kuadrant-system

authorino-cf4c9784b-2k5lg

Scheduled

Successfully assigned kuadrant-system/authorino-cf4c9784b-2k5lg to ip-10-0-141-154.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-6d9f996676-vjbcz to ip-10-0-139-154.ec2.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-dnhb4 to ip-10-0-141-154.ec2.internal

opendatahub

tls-scanner

Scheduled

Successfully assigned opendatahub/tls-scanner to ip-10-0-139-211.ec2.internal

opendatahub

odh-model-controller-769875db56-sd2xj

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-sd2xj to ip-10-0-139-154.ec2.internal

kuadrant-system

authorino-57f7f8659f-sntjd

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-sntjd to ip-10-0-141-154.ec2.internal

opendatahub

model-serving-api-75bd89fdfc-l55wq

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-l55wq to ip-10-0-139-154.ec2.internal

opendatahub

maas-discovery-6ccf9dc58d-lmbjd

Scheduled

Successfully assigned opendatahub/maas-discovery-6ccf9dc58d-lmbjd to ip-10-0-139-154.ec2.internal

opendatahub

maas-controller-7c9fb575f4-whbfb

Scheduled

Successfully assigned opendatahub/maas-controller-7c9fb575f4-whbfb to ip-10-0-141-154.ec2.internal

opendatahub

maas-controller-56f5cc5745-gvw6r

Scheduled

Successfully assigned opendatahub/maas-controller-56f5cc5745-gvw6r to ip-10-0-141-154.ec2.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-zvnd7

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-zvnd7 to ip-10-0-141-154.ec2.internal

opendatahub

llmisvc-controller-manager-6466c554d8-zhl5q

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-zhl5q to ip-10-0-141-154.ec2.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-pk7hk

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-pk7hk to ip-10-0-139-154.ec2.internal

opendatahub

kserve-controller-manager-776cc66d47-2fpp4

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-2fpp4 to ip-10-0-141-154.ec2.internal

kuadrant-system

authorino-5bffb98697-nmcs5

Scheduled

Successfully assigned kuadrant-system/authorino-5bffb98697-nmcs5 to ip-10-0-141-154.ec2.internal

opendatahub

ai-gateway-controller-5d8dddb676-99jw5

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-99jw5 to ip-10-0-139-211.ec2.internal

opendatahub

ai-gateway-controller-5d46c99c4f-tbkvl

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d46c99c4f-tbkvl to ip-10-0-139-211.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-c5ccw to ip-10-0-139-154.ec2.internal

kuadrant-system

authorino-6f6944bcdd-x26fl

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-x26fl to ip-10-0-141-154.ec2.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl to ip-10-0-139-154.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-rqchg to ip-10-0-141-154.ec2.internal

kuadrant-system

authorino-6889cbd8c4-s7dq6

Scheduled

Successfully assigned kuadrant-system/authorino-6889cbd8c4-s7dq6 to ip-10-0-141-154.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-fjwrw to ip-10-0-139-154.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-fwk6l to ip-10-0-141-154.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-gw4tw to ip-10-0-139-154.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-77bfb58f8c-ltt9g to ip-10-0-139-211.ec2.internal

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-55bb6c544c from 0 to 1

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-55bb6c544c

SuccessfulCreate

Created pod: router-default-55bb6c544c-dhlks
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x2)

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x3)

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

Created

Container created

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

Started

Container started

openshift-ingress

multus

router-default-55bb6c544c-dhlks

AddedInterface

Add eth0 [10.132.0.21/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-55bb6c544c-dhlks

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-b42wp

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-b42wp

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.402s (2.402s including waiting). Image size: 185476477 bytes.
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-6bd8768855 from 0 to 1

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-6bd8768855

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.085s (10.085s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

Started

Container started

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd8768855-g5hn5

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 10.396s (10.396s including waiting). Image size: 183555333 bytes.

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-4jntq

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: waiting for spec update of deployment "dns-operator-controller-manager" to be observed...

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-4jntq

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-4jntq

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-4jntq

Started

Container started

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-4jntq

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-4jntq

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 2.365s (2.365s including waiting). Image size: 190804149 bytes.

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-6d9f996676

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-6d9f996676-whcsd

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-6d9f996676

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-6d9f996676-vjbcz

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-6d9f996676

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-6d9f996676-xtvml

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-6d9f996676 from 0 to 3

kuadrant-system

dns-operator-controller-manager-8557cbdb84-4jntq_121c78c4-cac6-4239-819f-9ec9f6e6d111

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-4jntq_121c78c4-cac6-4239-819f-9ec9f6e6d111 became leader
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-6d9f996676-whcsd

AddedInterface

Add eth0 [10.133.0.17/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-6d9f996676-vjbcz

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-6d9f996676-xtvml

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-9mzhr

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Started

Container started

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.731s (2.731s including waiting). Image size: 273709574 bytes.

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-5jfpf

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.759s (2.759s including waiting). Image size: 273709574 bytes.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.757s (2.757s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 308ms (308ms including waiting). Image size: 273709574 bytes.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-5jfpf

AddedInterface

Add eth0 [10.133.0.18/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 638ms (638ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 264ms (264ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Created

Container created

opendatahub-operator-system

opendatahub-operator-controller-manager-6d9f996676-xtvml_f11c66a1-3aab-428e-98cb-510da286eb23

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-6d9f996676-xtvml_f11c66a1-3aab-428e-98cb-510da286eb23 became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-vjbcz

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-xtvml

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-5jfpf

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-6d9f996676-whcsd

Started

Container started

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

operator-lifecycle-manager

install-mfn6w

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-5jfpf

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 2.465s (2.465s including waiting). Image size: 224874686 bytes.
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-5jfpf

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-5jfpf

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-5jfpf_91563d85-aa0a-48cf-922d-6ff69322959f became leader

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

Started

Container started

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-9mzhr_7459dd13-63ad-432c-bc60-864257475449

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-9mzhr_7459dd13-63ad-432c-bc60-864257475449 became leader

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-n5j5c

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 4.058s (4.058s including waiting). Image size: 424244516 bytes.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

multus

authorino-operator-7b7bb4bc77-n5j5c

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-n5j5c

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"
(x3)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-c8z66

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-c8z66

AddedInterface

Add eth0 [10.133.0.19/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-c8z66

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-vtfw2

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-vtfw2

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-vtfw2

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-vtfw2

Started

Container started

kuadrant-system

limitador-operator-controller-manager-5778bb4449-vtfw2_b144126b-51a0-4f1e-b377-e0bb1dd872d6

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-vtfw2_b144126b-51a0-4f1e-b377-e0bb1dd872d6 became leader

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-vtfw2

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-vtfw2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 1.303s (1.303s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-c8z66

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 4.444s (4.444s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-c8z66

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-c8z66

Started

Container started

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-8lk25

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-8lk25

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-dspzv

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-dspzv

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-dspzv

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-8lk25

Started

Container started

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-8lk25
(x3)

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

FailedCreate

Error creating: pods "cert-manager-webhook-5dbbbf75cc-" is forbidden: error looking up service account cert-manager/cert-manager-webhook: serviceaccount "cert-manager-webhook" not found

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-8lk25

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-n5j5c

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 13.398s (13.398s including waiting). Image size: 167738465 bytes.
(x8)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-nnx7p

kuadrant-system

authorino-operator-7b7bb4bc77-n5j5c_8b35ba08-94b4-4b9a-8417-956765d464e2

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-n5j5c_8b35ba08-94b4-4b9a-8417-956765d464e2 became leader

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-nnx7p

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-nnx7p

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-nnx7p

Started

Container started

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-nnx7p

Created

Container created

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-nnx7p

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.134s (3.134s including waiting). Image size: 309650901 bytes.

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1
(x9)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-dspzv

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 8.673s (8.673s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-dspzv

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-dspzv

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-jrw8c

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

replicaset-controller

kube-auth-proxy-74f99fd9b4

SuccessfulCreate

Created pod: kube-auth-proxy-74f99fd9b4-jrw8c

openshift-ingress

multus

kube-auth-proxy-74f99fd9b4-jrw8c

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-74f99fd9b4 from 0 to 2

cert-manager

kubelet

cert-manager-597b95cb84-67rqd

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-ingress

multus

kube-auth-proxy-74f99fd9b4-854np

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

kube-auth-proxy-74f99fd9b4

SuccessfulCreate

Created pod: kube-auth-proxy-74f99fd9b4-854np

cert-manager

multus

cert-manager-597b95cb84-67rqd

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-68dd7b898f from 0 to 1

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-67rqd

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-68dd7b898f

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6

Started

Container started

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-68dd7b898fpjcb6

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-854np

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-sntjd

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0
(x3)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-9mzhr

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-9mzhr

Killing

Stopping container manager

kuadrant-system

multus

authorino-57f7f8659f-sntjd

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-57f7f8659f-sntjd

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-jrw8c

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-854np

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.521s (2.521s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-854np

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-854np

Started

Container started

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-jrw8c

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.708s (2.708s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-74f99fd9b4-jrw8c

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-sntjd

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-sntjd

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-sntjd

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.732s (2.732s including waiting). Image size: 204096810 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pk7hk

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-pk7hk

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-pk7hk

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

cert-manager

kubelet

cert-manager-597b95cb84-67rqd

Started

Container started

cert-manager

kubelet

cert-manager-597b95cb84-67rqd

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 10.936s (10.936s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-597b95cb84-67rqd

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pk7hk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 2.254s (2.254s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pk7hk

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pk7hk

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-5dj2h"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-swsc5"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-67b799d75 from 0 to 1

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-67b799d75

SuccessfulCreate

Created pod: jobset-controller-manager-67b799d75-xbgnr

openshift-jobset-operator

kubelet

jobset-controller-manager-67b799d75-xbgnr

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-jobset-operator

multus

jobset-controller-manager-67b799d75-xbgnr

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-8lk25_bd4d07c4-ebaf-47c4-85d1-932d6d22361b

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-8lk25_bd4d07c4-ebaf-47c4-85d1-932d6d22361b became leader

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-l7nkf

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-vn568

kuadrant-system

kubelet

authorino-56c6c9ddd5-l7nkf

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 871ms (871ms including waiting). Image size: 204096810 bytes.

kuadrant-system

multus

limitador-limitador-74dfcfb894-vn568

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-zznzj

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

multus

authorino-56c6c9ddd5-l7nkf

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-vn568

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

kubelet

authorino-56c6c9ddd5-l7nkf

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-56c6c9ddd5-l7nkf

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-l7nkf

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-zznzj

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-sntjd

kuadrant-system

multus

limitador-limitador-7c9b87d574-zznzj

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

authorino-57f7f8659f-sntjd

Killing

Stopping container authorino

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-zznzj

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-vn568

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 1.969s (1.969s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-vn568

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-vn568

Started

Container started

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-zznzj

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 1.2s (1.2s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-zznzj

Started

Container started

openshift-jobset-operator

kubelet

jobset-controller-manager-67b799d75-xbgnr

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 6.76s (6.76s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-67b799d75-xbgnr

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-67b799d75-xbgnr

Started

Container started

opendatahub

kserve-module-controller-manager-7566cf5fc8-pk7hk_6ba09b44-0ee8-49c4-b111-fad6c929fd7a

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-pk7hk_6ba09b44-0ee8-49c4-b111-fad6c929fd7a became leader

openshift-jobset-operator

jobset-controller-manager-67b799d75-xbgnr_77a92018-2774-4d5d-ba61-4dd8b5ccc3ca

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-67b799d75-xbgnr_77a92018-2774-4d5d-ba61-4dd8b5ccc3ca became leader

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-vn568

Killing

Stopping container limitador

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-vn568
(x3)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-56f5cc5745 from 0 to 1

opendatahub

replicaset-controller

maas-controller-56f5cc5745

SuccessfulCreate

Created pod: maas-controller-56f5cc5745-gvw6r

opendatahub

multus

maas-controller-56f5cc5745-gvw6r

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-56f5cc5745-gvw6r

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:9cff422470642d0aa4474c1f8d86e25af7be12d3f0207d1a0545134ef35ddaeb"

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-7c9fb575f4 from 0 to 1

opendatahub

replicaset-controller

maas-controller-7c9fb575f4

SuccessfulCreate

Created pod: maas-controller-7c9fb575f4-whbfb

opendatahub

multus

maas-controller-7c9fb575f4-whbfb

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

Created

Container created

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-8bzhx

opendatahub

kubelet

maas-controller-7c9fb575f4-whbfb

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:9cff422470642d0aa4474c1f8d86e25af7be12d3f0207d1a0545134ef35ddaeb"

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

Started

Container started

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-zvnd7

FailedMount

MountVolume.SetUp failed for volume "cert" : secret "llmisvc-webhook-server-cert" not found

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-2fpp4

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-zvnd7

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-sd2xj

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-l55wq

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-8bzhx

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

opendatahub

kubelet

model-serving-api-75bd89fdfc-l55wq

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-zvnd7

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-b42wp

opendatahub

multus

odh-model-controller-769875db56-sd2xj

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

opendatahub

multus

model-serving-api-75bd89fdfc-l55wq

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-zvnd7

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-b42wp

Killing

Stopping container discovery

opendatahub

multus

kserve-controller-manager-776cc66d47-2fpp4

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

opendatahub

kubelet

kserve-controller-manager-776cc66d47-2fpp4

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

kubelet

odh-model-controller-769875db56-sd2xj

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

kubelet

maas-controller-7c9fb575f4-whbfb

Started

Container started

opendatahub

kubelet

maas-controller-56f5cc5745-gvw6r

Created

Container created

opendatahub

kubelet

maas-controller-56f5cc5745-gvw6r

Started

Container started

opendatahub

kubelet

maas-controller-56f5cc5745-gvw6r

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:9cff422470642d0aa4474c1f8d86e25af7be12d3f0207d1a0545134ef35ddaeb" in 3.157s (3.157s including waiting). Image size: 200095258 bytes.

opendatahub

kubelet

maas-controller-7c9fb575f4-whbfb

Created

Container created

opendatahub

kubelet

maas-controller-7c9fb575f4-whbfb

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:9cff422470642d0aa4474c1f8d86e25af7be12d3f0207d1a0545134ef35ddaeb" in 3.104s (3.104s including waiting). Image size: 200095258 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-2fpp4

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 2.247s (2.247s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-zvnd7

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.243s (2.243s including waiting). Image size: 235100981 bytes.

opendatahub

maas-controller-56f5cc5745-gvw6r_27e7c3ad-ef51-44a9-af01-54e8989a1fc4

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-56f5cc5745-gvw6r_27e7c3ad-ef51-44a9-af01-54e8989a1fc4 became leader

opendatahub

kubelet

model-serving-api-75bd89fdfc-l55wq

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-l55wq

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-2fpp4

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-2fpp4

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-l55wq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 1.925s (1.925s including waiting). Image size: 184925761 bytes.

opendatahub

odh-model-controller-769875db56-sd2xj_fe28c31e-c277-41da-95c3-5e5b9e0a0b5e

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-sd2xj_fe28c31e-c277-41da-95c3-5e5b9e0a0b5e became leader

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-zvnd7

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-zvnd7

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-sd2xj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 2.733s (2.733s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-sd2xj

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-sd2xj

Created

Container created

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

opendatahub

kserve-controller-manager-776cc66d47-2fpp4_0deea4ba-c847-466e-839b-7b95e80a03dd

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-2fpp4_0deea4ba-c847-466e-839b-7b95e80a03dd became leader

opendatahub

llmisvc-controller-manager-869d5dcc5d-zvnd7_ed86f49c-08fb-4d59-95a7-ccaf6c7228db

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-zvnd7_ed86f49c-08fb-4d59-95a7-ccaf6c7228db became leader

opendatahub

deployment-controller

maas-discovery

ScalingReplicaSet

Scaled up replica set maas-discovery-6ccf9dc58d from 0 to 1
(x3)

opendatahub

replicaset-controller

maas-discovery-6ccf9dc58d

FailedCreate

Error creating: pods "maas-discovery-6ccf9dc58d-" is forbidden: error looking up service account opendatahub/maas-discovery: serviceaccount "maas-discovery" not found

opendatahub

kubelet

maas-discovery-6ccf9dc58d-lmbjd

Pulling

Pulling image "quay.io/opendatahub/odh-maas-discovery@sha256:bb5dd3a1d77988be38d66dd09fbc6f53b0ffb58cfc2fd8dfb670ae3439079d75"

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

opendatahub

multus

maas-discovery-6ccf9dc58d-lmbjd

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

opendatahub

replicaset-controller

maas-discovery-6ccf9dc58d

SuccessfulCreate

Created pod: maas-discovery-6ccf9dc58d-lmbjd

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

kubelet

maas-discovery-6ccf9dc58d-lmbjd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-maas-discovery@sha256:bb5dd3a1d77988be38d66dd09fbc6f53b0ffb58cfc2fd8dfb670ae3439079d75" in 2.152s (2.152s including waiting). Image size: 162110345 bytes.

opendatahub

kubelet

maas-discovery-6ccf9dc58d-lmbjd

Created

Container created

opendatahub

kubelet

maas-discovery-6ccf9dc58d-lmbjd

Started

Container started

opendatahub

kubelet

maas-controller-56f5cc5745-gvw6r

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-56f5cc5745

SuccessfulDelete

Deleted pod: maas-controller-56f5cc5745-gvw6r

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-56f5cc5745 from 1 to 0

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-dnhb4

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 10.849s (10.849s including waiting). Image size: 1089502687 bytes.

opendatahub

maas-controller-7c9fb575f4-whbfb_b67ed5ac-5e73-460f-985f-0b45a1c61b75

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-7c9fb575f4-whbfb_b67ed5ac-5e73-460f-985f-0b45a1c61b75 became leader
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-x26fl

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

kubelet

authorino-6889cbd8c4-s7dq6

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-x26fl

kuadrant-system

multus

authorino-6889cbd8c4-s7dq6

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6889cbd8c4 from 0 to 1

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulCreate

Created pod: authorino-6889cbd8c4-s7dq6
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Started

Container started
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

authorino-6889cbd8c4-s7dq6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.296s (2.296s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6889cbd8c4-s7dq6

Started

Container started

kuadrant-system

kubelet

authorino-6889cbd8c4-s7dq6

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-l7nkf

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-l7nkf

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Unhealthy

Startup probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

ProbeError

Startup probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kubelet

authorino-cf4c9784b-2k5lg

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-cf4c9784b from 0 to 1

kuadrant-system

replicaset-controller

authorino-cf4c9784b

SuccessfulCreate

Created pod: authorino-cf4c9784b-2k5lg

kuadrant-system

multus

authorino-cf4c9784b-2k5lg

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-cf4c9784b-2k5lg

Started

Container started

kuadrant-system

kubelet

authorino-cf4c9784b-2k5lg

Created

Container created

kuadrant-system

kubelet

authorino-cf4c9784b-2k5lg

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 490ms (490ms including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-6889cbd8c4-s7dq6

Killing

Stopping container authorino

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6889cbd8c4 from 1 to 0

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulDelete

Deleted pod: authorino-6889cbd8c4-s7dq6

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-zhl5q

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-zhl5q

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

multus

llmisvc-controller-manager-6466c554d8-zhl5q

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-zhl5q

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-zhl5q

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-zhl5q

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 383ms (383ms including waiting). Image size: 235100981 bytes.

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-zvnd7

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-zvnd7

Killing

Stopping container manager

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-99jw5

opendatahub

multus

ai-gateway-controller-5d8dddb676-99jw5

AddedInterface

Add eth0 [10.134.0.28/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-99jw5

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-99jw5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 2.317s (2.318s including waiting). Image size: 183979430 bytes.

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-99jw5

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-99jw5

Created

Container created

opendatahub

ai-gateway-controller-5d8dddb676-99jw5_5abfd914-cd15-4e51-a53f-d20dc4f68f16

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-99jw5_5abfd914-cd15-4e51-a53f-d20dc4f68f16 became leader
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulCreate

Created pod: payload-processing-55fbbbf9dc-bhvqh
(x5)

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

FailedCreate

Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-cmbvc

openshift-ingress

multus

payload-processing-55fbbbf9dc-bhvqh

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-bhvqh

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-7b98bb678c-cmbvc

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-cmbvc

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

opendatahub

llmisvc-controller-manager-6466c554d8-zhl5q_c291e339-0c51-49e4-97c3-e01c7f2ebf92

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-zhl5q_c291e339-0c51-49e4-97c3-e01c7f2ebf92 became leader

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-bhvqh

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-cmbvc

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-cmbvc

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.918s (1.918s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-bhvqh

Started

Container started

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-bhvqh

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.967s (1.967s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-cmbvc

Created

Container created

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d46c99c4f from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-5d46c99c4f

SuccessfulCreate

Created pod: ai-gateway-controller-5d46c99c4f-tbkvl

opendatahub

multus

ai-gateway-controller-5d46c99c4f-tbkvl

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-5d46c99c4f-tbkvl

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d46c99c4f-tbkvl

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 287ms (287ms including waiting). Image size: 183979430 bytes.

opendatahub

kubelet

ai-gateway-controller-5d46c99c4f-tbkvl

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d46c99c4f-tbkvl

Created

Container created

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-99jw5

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-99jw5

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.22s (1.22s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-vnlbw

Started

Container started

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-46lnp

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-46lnp

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-46lnp

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-56bs5

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-56bs5

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-46lnp

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.282s (1.282s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-56bs5

Created

Container created

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-56bs5

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 228ms (228ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-46lnp

Created

Container created

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-56bs5

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-56bs5

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-46lnp

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-r99sj

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-r99sj

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-r99sj

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-r99sj

Created

Container created

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-r99sj

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 274ms (274ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-r99sj

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

Started

Container started

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 268ms (268ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8srqmx

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

opendatahub

ai-gateway-controller-5d46c99c4f-tbkvl_97dae541-6a99-45d3-a566-fbcb9b42ea85

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d46c99c4f-tbkvl_97dae541-6a99-45d3-a566-fbcb9b42ea85 became leader

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-qpzbq

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

kubelet

echo-simulator-kserve-64b6f8f96c-qpzbq

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

multus

echo-simulator-kserve-64b6f8f96c-qpzbq

AddedInterface

Add eth0 [10.132.0.54/23] from ovn-kubernetes

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

kubelet

echo-simulator-kserve-64b6f8f96c-qpzbq

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 1.992s (1.992s including waiting). Image size: 78208878 bytes.

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-s54dv

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 337ms (337ms including waiting). Image size: 98346788 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-qpzbq

Created

Container created

llm

kubelet

echo-simulator-kserve-64b6f8f96c-qpzbq

Started

Container started

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 220ms (220ms including waiting). Image size: 98346788 bytes.

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-2tksl

Started

Container started

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-r7bct

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 1 to 4

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.807s (2.807s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Started

Container started

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

kuadrant-system

replicaset-controller

authorino-5bffb98697

SuccessfulCreate

Created pod: authorino-5bffb98697-nmcs5

kuadrant-system

kubelet

authorino-5bffb98697-nmcs5

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-5bffb98697-nmcs5

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-5bffb98697-nmcs5

Created

Container created
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

Unhealthy

Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-cf4c9784b from 1 to 0

kuadrant-system

kubelet

authorino-5bffb98697-nmcs5

Started

Container started
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

ProbeError

Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused body:

kuadrant-system

replicaset-controller

authorino-cf4c9784b

SuccessfulDelete

Deleted pod: authorino-cf4c9784b-2k5lg

kuadrant-system

kubelet

authorino-5bffb98697-nmcs5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 661ms (661ms including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-cf4c9784b-2k5lg

Killing

Stopping container authorino

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 7.684s (7.685s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

ProbeError

Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Unhealthy

Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-rqchg

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-gw4tw
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Unhealthy

Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

ProbeError

Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-77bfb58f8c from 4 to 7

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

Unhealthy

Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

Unhealthy

Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

ProbeError

Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

Unhealthy

Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

ProbeError

Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-rqchg

ProbeError

Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:

opendatahub

kubelet

tls-scanner

Pulling

Pulling image "registry.ci.openshift.org/ocp/4.22:tls-scanner-tool"

opendatahub

multus

tls-scanner

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

opendatahub

kubelet

tls-scanner

Created

Container created

opendatahub

kubelet

tls-scanner

Pulled

Successfully pulled image "registry.ci.openshift.org/ocp/4.22:tls-scanner-tool" in 10.03s (10.03s including waiting). Image size: 1003292452 bytes.

opendatahub

kubelet

tls-scanner

Started

Container started
(x2)

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-n5j5c

Started

Container started
(x2)

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-n5j5c

Created

Container created

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-n5j5c

Pulled

Container image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" already present on machine and can be accessed by the pod

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

authorino-operator-7b7bb4bc77-n5j5c_22b4cf38-a855-43c4-b2dc-cb53e0f2b820

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-n5j5c_22b4cf38-a855-43c4-b2dc-cb53e0f2b820 became leader

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-77bfb58f8c from 7 to 2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fjwrw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-r7bct

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-c5ccw

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-gw4tw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-fwk6l

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-r7bct

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-77bfb58f8c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-77bfb58f8c from 2 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-77bfb58f8c-ltt9g

Killing

Stopping container istio-proxy