Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-7lxxs to ip-10-0-143-171.ec2.internal

ai-tenants

e2e-autores-6653eb

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-autores-6653eb: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-explicit-4a0349

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-explicit-4a0349: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-331c9e

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-331c9e: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-b4361f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-b4361f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-a-5b97f1

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-a-5b97f1: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-b-e759e5

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-b-e759e5: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-a-72b663

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-a-72b663: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-b-131649

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-b-131649: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-4a67e25f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-4a67e25f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-7b218f2f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-7b218f2f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-a-4429aa

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-a-4429aa: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-b-0a0ec6

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-b-0a0ec6: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w4-1ffaa0

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w4-1ffaa0: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w4-2d0da2

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w4-2d0da2: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-86c2e2

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-86c2e2: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-156ae1

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-156ae1: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-d2m4k

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-d2m4k to ip-10-0-128-190.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-867995fc99-vnxfr

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-867995fc99-vnxfr to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs to ip-10-0-130-191.ec2.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-mf4wk

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-mf4wk to ip-10-0-130-191.ec2.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-867995fc99-zzfz8

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-867995fc99-zzfz8 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq to ip-10-0-130-191.ec2.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-wzr8g

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-wzr8g to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-689fc7c65b-m8nsp

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-689fc7c65b-m8nsp to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-689fc7c65b-4wrkw

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-689fc7c65b-4wrkw to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5 to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-65b496c9b-pbkxk

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-65b496c9b-pbkxk to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-64f7c4d6bf-hddgq

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-64f7c4d6bf-hddgq to ip-10-0-128-190.ec2.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-9z5kt

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-9z5kt to ip-10-0-128-190.ec2.internal

openshift-ingress

router-default-6c7f54c9b7-t6wbf

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-6c7f54c9b7-t6wbf

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-6c7f54c9b7-t6wbf

Scheduled

Successfully assigned openshift-ingress/router-default-6c7f54c9b7-t6wbf to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb to ip-10-0-130-191.ec2.internal

llm

e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

Scheduled

Successfully assigned llm/e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-75cb49bc4f-m5hf9

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-75cb49bc4f-m5hf9 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88 to ip-10-0-130-191.ec2.internal

models-as-a-service

payload-processing-external-model-5bcd7997-l5wbj

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5bcd7997-l5wbj to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4 to ip-10-0-130-191.ec2.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-85d774877-jtx2p

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-85d774877-jtx2p to ip-10-0-128-190.ec2.internal

openshift-jobset-operator

jobset-controller-manager-58f856b7f7-px6s2

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-58f856b7f7-px6s2 to ip-10-0-143-171.ec2.internal

kuadrant-system

authorino-56c6c9ddd5-xvz6f

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-xvz6f to ip-10-0-128-190.ec2.internal

opendatahub

test-curl-7316-4069cb

Scheduled

Successfully assigned opendatahub/test-curl-7316-4069cb to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc to ip-10-0-130-191.ec2.internal

opendatahub

test-curl-7316-83dcde

Scheduled

Successfully assigned opendatahub/test-curl-7316-83dcde to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6 to ip-10-0-128-190.ec2.internal

opendatahub

odh-model-controller-769875db56-nmtcb

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-nmtcb to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6 to ip-10-0-130-191.ec2.internal

kuadrant-system

authorino-57f7f8659f-cllp7

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-cllp7 to ip-10-0-128-190.ec2.internal

opendatahub

model-serving-api-75bd89fdfc-78fsn

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-78fsn to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds to ip-10-0-128-190.ec2.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-r9t9j to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6 to ip-10-0-130-191.ec2.internal

kuadrant-system

authorino-5ffb77fcd4-8kmbf

Scheduled

Successfully assigned kuadrant-system/authorino-5ffb77fcd4-8kmbf to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482 to ip-10-0-130-191.ec2.internal

kuadrant-system

authorino-6889cbd8c4-2p2jc

Scheduled

Successfully assigned kuadrant-system/authorino-6889cbd8c4-2p2jc to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c to ip-10-0-128-190.ec2.internal

kuadrant-system

authorino-6f6944bcdd-lrwvj

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-lrwvj to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t to ip-10-0-128-190.ec2.internal

kuadrant-system

authorino-948677c5f-lc4kc

Scheduled

Successfully assigned kuadrant-system/authorino-948677c5f-lc4kc to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Scheduled

Successfully assigned openshift-ingress/e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

Scheduled

Successfully assigned openshift-ingress/e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd to ip-10-0-128-190.ec2.internal

kuadrant-system

authorino-operator-7b7bb4bc77-jf2qh

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-jf2qh to ip-10-0-143-171.ec2.internal

models-as-a-service

payload-processing-external-model-75cb49bc4f-gj7p4

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-75cb49bc4f-gj7p4 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

Scheduled

Successfully assigned openshift-ingress/e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2 to ip-10-0-130-191.ec2.internal

ai-tenants

e2e-api-a-cb587c

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-a-cb587c: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-6d2jm to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Scheduled

Successfully assigned openshift-ingress/e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-7g4cr to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-7jwxs to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s to ip-10-0-128-190.ec2.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz to ip-10-0-143-171.ec2.internal

opendatahub

maas-controller-6dfdc587c5-vbgqs

Scheduled

Successfully assigned opendatahub/maas-controller-6dfdc587c5-vbgqs to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Scheduled

Successfully assigned openshift-ingress/e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-noten-682a07-openshift-default-85774c99b9-hrdms

Scheduled

Successfully assigned openshift-ingress/e2e-noten-682a07-openshift-default-85774c99b9-hrdms to ip-10-0-130-191.ec2.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-t7s86

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-t7s86 to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

Scheduled

Successfully assigned openshift-ingress/e2e-ait-c1daa315-openshift-default-87b646887-fxnt9 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4 to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-7x8hv to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-8jh2s to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Scheduled

Successfully assigned openshift-ingress/e2e-ait-cbd38938-openshift-default-55744f6757-mdddk to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4 to ip-10-0-130-191.ec2.internal

llm

e2e-same-id-a-0e21d4af

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-b-0e21d4af

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-92rg7 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8 to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql to ip-10-0-130-191.ec2.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-k9lp2

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-k9lp2 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5 to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-9qpkm to ip-10-0-143-171.ec2.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9 to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-bzkwg to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq to ip-10-0-130-191.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-5rnng

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-5rnng to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92 to ip-10-0-128-190.ec2.internal

opendatahub

test-curl-7319-1c0cee

Scheduled

Successfully assigned opendatahub/test-curl-7319-1c0cee to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d to ip-10-0-130-191.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-82pkj

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-82pkj to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-c8dts to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-cthxb to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-dzkt9 to ip-10-0-130-191.ec2.internal

openshift-ingress

kube-auth-proxy-76667dfcdc-6cmzw

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-76667dfcdc-6cmzw to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-lnxjp to ip-10-0-130-191.ec2.internal

opendatahub

maas-controller-6dfdc587c5-jl224

Scheduled

Successfully assigned opendatahub/maas-controller-6dfdc587c5-jl224 to ip-10-0-130-191.ec2.internal

opendatahub

maas-controller-6dfdc587c5-jcn8k

Scheduled

Successfully assigned opendatahub/maas-controller-6dfdc587c5-jcn8k to ip-10-0-130-191.ec2.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-5zmt7 to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-lpzqg to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

Scheduled

Successfully assigned openshift-ingress/e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5 to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-8r2q9 to ip-10-0-143-171.ec2.internal

ai-tenants

e2e-api-b-2e8614

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-b-2e8614: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd to ip-10-0-130-191.ec2.internal

opendatahub

maas-controller-64678f5d5d-96mcs

Scheduled

Successfully assigned opendatahub/maas-controller-64678f5d5d-96mcs to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-mcrdq to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v to ip-10-0-143-171.ec2.internal

models-as-a-service

payload-processing-external-model-7cd8fc5c4-v9dth

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7cd8fc5c4-v9dth to ip-10-0-130-191.ec2.internal

opendatahub

test-curl-7319-2f122b

Scheduled

Successfully assigned opendatahub/test-curl-7319-2f122b to ip-10-0-143-171.ec2.internal

kuadrant-system

limitador-limitador-74dfcfb894-ff8bx

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-ff8bx to ip-10-0-130-191.ec2.internal

opendatahub

test-curl-7322-03bdd6

Scheduled

Successfully assigned opendatahub/test-curl-7322-03bdd6 to ip-10-0-143-171.ec2.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-75cwz

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-75cwz to ip-10-0-143-171.ec2.internal

openshift-ingress

kube-auth-proxy-76667dfcdc-rj9l9

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-76667dfcdc-rj9l9 to ip-10-0-130-191.ec2.internal

opendatahub

test-curl-7322-68ae22

Scheduled

Successfully assigned opendatahub/test-curl-7322-68ae22 to ip-10-0-143-171.ec2.internal

opendatahub

llmisvc-controller-manager-6466c554d8-54d5w

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-54d5w to ip-10-0-143-171.ec2.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-qdflj

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-qdflj to ip-10-0-130-191.ec2.internal

opendatahub

test-curl-7322-843fbc

Scheduled

Successfully assigned opendatahub/test-curl-7322-843fbc to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6 to ip-10-0-143-171.ec2.internal

kuadrant-system

limitador-limitador-7c9b87d574-f8vvn

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-f8vvn to ip-10-0-128-190.ec2.internal

opendatahub

kserve-controller-manager-776cc66d47-4ggmd

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-4ggmd to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5 to ip-10-0-143-171.ec2.internal

opendatahub

test-curl-7322-8e57e7

Scheduled

Successfully assigned opendatahub/test-curl-7322-8e57e7 to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-2p5f4 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw to ip-10-0-128-190.ec2.internal

opendatahub

test-curl-7322-9e30f2

Scheduled

Successfully assigned opendatahub/test-curl-7322-9e30f2 to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr to ip-10-0-130-191.ec2.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-569f5

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-569f5 to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-qbxzw to ip-10-0-130-191.ec2.internal

llm

echo-simulator-kserve-64b6f8f96c-c242s

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-c242s to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-qpbjh to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

Scheduled

Successfully assigned openshift-ingress/e2e-autores-6653eb-openshift-default-6df94d5785-njjtz to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-r88l9 to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-rr46x to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-95b747dcd-rw869

Scheduled

Successfully assigned openshift-ingress/payload-processing-95b747dcd-rw869 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-5985fbdcbc-h9tqm

Scheduled

Successfully assigned openshift-ingress/payload-processing-5985fbdcbc-h9tqm to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-t7kz9 to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-wcq9z to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-wxgc4 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Scheduled

Successfully assigned openshift-ingress/e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-processing-55fbbbf9dc-r9bfq

Scheduled

Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-r9bfq to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-5btvs to ip-10-0-130-191.ec2.internal

opendatahub

ai-gateway-controller-5f464594c6-562pp

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5f464594c6-562pp to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h to ip-10-0-128-190.ec2.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf to ip-10-0-128-190.ec2.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4 to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28 to ip-10-0-143-171.ec2.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-d77tr

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-d77tr to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-7b98bb678c-b9jkn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-b9jkn to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-76bfcf8f7d-tzw9j

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-76bfcf8f7d-tzw9j to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-559f69b56-h5sc2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-559f69b56-h5sc2 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-xrpdz to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-6d8799cd9-tnpcj

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6d8799cd9-tnpcj to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2 to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9 to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-z57r2 to ip-10-0-128-190.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-z6fd6 to ip-10-0-128-190.ec2.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-pbhb4 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl to ip-10-0-128-190.ec2.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t to ip-10-0-128-190.ec2.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-mk4zz

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-mk4zz to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9 to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-z7p28 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6 to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm to ip-10-0-130-191.ec2.internal

llm

e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

Scheduled

Successfully assigned llm/e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt to ip-10-0-143-171.ec2.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw to ip-10-0-130-191.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-5h7r7 to ip-10-0-143-171.ec2.internal

openshift-ingress

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-55f85cf5d7-675z5 to ip-10-0-128-190.ec2.internal

opendatahub

test-curl-7322-a13e07

Scheduled

Successfully assigned opendatahub/test-curl-7322-a13e07 to ip-10-0-143-171.ec2.internal

opendatahub

test-curl-7322-b87bb1

Scheduled

Successfully assigned opendatahub/test-curl-7322-b87bb1 to ip-10-0-143-171.ec2.internal

llm

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Scheduled

Successfully assigned llm/e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7 to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc to ip-10-0-143-171.ec2.internal

openshift-ingress

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4 to ip-10-0-143-171.ec2.internal

opendatahub

ai-gateway-controller-5d8dddb676-wzdx8

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-5d8dddb676-wzdx8 to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z to ip-10-0-128-190.ec2.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-m6wvl

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-m6wvl to ip-10-0-130-191.ec2.internal

llm

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

Scheduled

Successfully assigned llm/e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq to ip-10-0-128-190.ec2.internal

llm

e2e-same-id-b-0e21d4af

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-a-0e21d4af

llm

e2e-same-id-b-0e21d4af

ModelNameConflictResolved

Model identity is no longer shared with any other MaaSModelRef in this namespace

opendatahub-operator-system

opendatahub-operator-controller-manager-867995fc99-8txkw

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-867995fc99-8txkw to ip-10-0-130-191.ec2.internal

openshift-ingress

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv to ip-10-0-128-190.ec2.internal

openshift-ingress

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q to ip-10-0-128-190.ec2.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-kjghr

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-kjghr to ip-10-0-128-190.ec2.internal

openshift-ingress

payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv to ip-10-0-130-191.ec2.internal

openshift-ingress

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b to ip-10-0-128-190.ec2.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-tr28v

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-tr28v to ip-10-0-128-190.ec2.internal

models-as-a-service

payload-processing-external-model-bf9485bb6-jrz9d

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-bf9485bb6-jrz9d to ip-10-0-128-190.ec2.internal

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-6c7f54c9b7 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-6c7f54c9b7

SuccessfulCreate

Created pod: router-default-6c7f54c9b7-t6wbf
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x2)

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x3)

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

Created

Container created

openshift-ingress

multus

router-default-6c7f54c9b7-t6wbf

AddedInterface

Add eth0 [10.132.0.9/23] from ovn-kubernetes

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

Started

Container started

openshift-ingress

kubelet

router-default-6c7f54c9b7-t6wbf

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-mf4wk

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-mf4wk

AddedInterface

Add eth0 [10.134.0.18/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.075s (3.075s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-6bd54bbbf9 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-675z5

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 0 to 1

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-675z5

AddedInterface

Add eth0 [10.133.0.17/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

AddedInterface

Add eth0 [10.133.0.19/23] from ovn-kubernetes

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.476s (3.476s including waiting). Image size: 183555333 bytes.
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-mk4zz

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-mk4zz

AddedInterface

Add eth0 [10.133.0.20/23] from ovn-kubernetes

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 20.49s (20.49s including waiting). Image size: 183555333 bytes.

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-mk4zz

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6bd54bbbf9-mdj9t

Started

Container started
(x2)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-mk4zz

Created

Container created

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-mk4zz

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-mk4zz

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 2.301s (2.301s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-mk4zz_78824e5a-900f-4dfb-be23-5b6e43b08037 became leader

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-t7s86

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-t7s86

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-t7s86

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

dns-operator-controller-manager-8557cbdb84-t7s86_4ea61a02-5356-4fa1-af6a-3657a3566257

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-t7s86_4ea61a02-5356-4fa1-af6a-3657a3566257 became leader

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-t7s86

Started

Container started

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-t7s86

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 2.619s (2.619s including waiting). Image size: 190804149 bytes.

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-t7s86

Created

Container created

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-5zmt7

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-9z5kt

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-9z5kt

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found
(x9)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-9z5kt

AddedInterface

Add eth0 [10.133.0.21/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-wzr8g

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-wzr8g

AddedInterface

Add eth0 [10.133.0.22/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-wzr8g

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-jf2qh

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-5zmt7_acf2cc7b-89fa-4805-bbb3-9c426df7e425

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-5zmt7_acf2cc7b-89fa-4805-bbb3-9c426df7e425 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-jf2qh

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 3.945s (3.945s including waiting). Image size: 424244516 bytes.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

multus

authorino-operator-7b7bb4bc77-jf2qh

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes
(x2)

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-wzr8g

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-9z5kt

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.918s (4.919s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-wzr8g

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.192s (3.192s including waiting). Image size: 309650901 bytes.

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-wzr8g

Created

Container created

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-9z5kt

Started

Container started

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-9z5kt

Created

Container created

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-jf2qh

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 2.257s (2.257s including waiting). Image size: 167738465 bytes.

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-jf2qh

Started

Container started

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-jf2qh

Created

Container created

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-k9lp2

AddedInterface

Add eth0 [10.134.0.19/23] from ovn-kubernetes

kuadrant-system

authorino-operator-7b7bb4bc77-jf2qh_7db4faa4-be6e-4a81-9da5-ffd4c145d147

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-jf2qh_7db4faa4-be6e-4a81-9da5-ffd4c145d147 became leader

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-867995fc99 from 0 to 3

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-867995fc99

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-867995fc99-zzfz8

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-k9lp2

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-867995fc99

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-867995fc99-8txkw

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-867995fc99

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-867995fc99-vnxfr

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-569f5

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-867995fc99-zzfz8

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-867995fc99-vnxfr

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k9lp2

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-867995fc99-8txkw

AddedInterface

Add eth0 [10.134.0.20/23] from ovn-kubernetes

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-569f5

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-569f5

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.
(x10)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-569f5

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 1.565s (1.565s including waiting). Image size: 177967631 bytes.

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-569f5

Created

Container created

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-569f5

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Created

Container created

kuadrant-system

limitador-operator-controller-manager-5778bb4449-569f5_800385c6-5a8b-4ad6-9ded-e5a3f120703f

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-569f5_800385c6-5a8b-4ad6-9ded-e5a3f120703f became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.379s (3.379s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.732s (2.732s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-d2m4k

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 2.903s (2.903s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Created

Container created

opendatahub-operator-system

opendatahub-operator-controller-manager-867995fc99-vnxfr_78b6f6b5-c51f-4f12-8bcd-272ff9cf6671

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-867995fc99-vnxfr_78b6f6b5-c51f-4f12-8bcd-272ff9cf6671 became leader

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Started

Container started

cert-manager

multus

cert-manager-597b95cb84-d2m4k

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-d2m4k

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

cert-manager

kubelet

cert-manager-597b95cb84-d2m4k

Created

Container created

cert-manager

kubelet

cert-manager-597b95cb84-d2m4k

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 320ms (320ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 305ms (305ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-8txkw

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 275ms (275ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-zzfz8

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-867995fc99-vnxfr

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k9lp2

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 5.292s (5.292s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k9lp2

Created

Container created

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-k9lp2

Started

Container started
(x4)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv
(x4)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-82pkj

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-82pkj

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-82pkj

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-82pkj

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-82pkj

Started

Container started

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-5zmt7

Killing

Stopping container manager

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-5zmt7

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-85d774877 from 0 to 1

openshift-ingress

replicaset-controller

kube-auth-proxy-76667dfcdc

SuccessfulCreate

Created pod: kube-auth-proxy-76667dfcdc-6cmzw

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-76667dfcdc from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-76667dfcdc

SuccessfulCreate

Created pod: kube-auth-proxy-76667dfcdc-rj9l9

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-85d774877

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-85d774877-jtx2p

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-72mrl"

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-rjvcl"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-85d774877-jtx2p

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-85d774877-jtx2p

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

kube-auth-proxy-76667dfcdc-6cmzw

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-85d774877-jtx2p

Created

Container created

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-6cmzw

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-76667dfcdc-rj9l9

AddedInterface

Add eth0 [10.134.0.21/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-rj9l9

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-85d774877-jtx2p

Started

Container started

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-rj9l9

Created

Container created

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-rj9l9

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-6cmzw

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-rj9l9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.419s (2.419s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-6cmzw

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-58f856b7f7 from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-76667dfcdc-6cmzw

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.363s (2.363s including waiting). Image size: 188904677 bytes.

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-58f856b7f7

SuccessfulCreate

Created pod: jobset-controller-manager-58f856b7f7-px6s2

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-qdflj

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

openshift-jobset-operator

multus

jobset-controller-manager-58f856b7f7-px6s2

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-58f856b7f7-px6s2

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-qdflj

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-qdflj

AddedInterface

Add eth0 [10.134.0.22/23] from ovn-kubernetes

openshift-jobset-operator

kubelet

jobset-controller-manager-58f856b7f7-px6s2

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-qdflj

Started

Container started

openshift-jobset-operator

kubelet

jobset-controller-manager-58f856b7f7-px6s2

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 3.275s (3.275s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-58f856b7f7-px6s2

Started

Container started

openshift-jobset-operator

jobset-controller-manager-58f856b7f7-px6s2_f9b1d031-bdc1-4a2e-a05d-3d54ba31df59

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-58f856b7f7-px6s2_f9b1d031-bdc1-4a2e-a05d-3d54ba31df59 became leader

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-qdflj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.745s (1.745s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-qdflj

Created

Container created

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-82pkj_ffddf328-90a0-46d9-809c-634bc49eabc5

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-82pkj_ffddf328-90a0-46d9-809c-634bc49eabc5 became leader

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ff8bx

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

multus

limitador-limitador-74dfcfb894-ff8bx

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-ff8bx

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

kuadrant-system

kubelet

authorino-57f7f8659f-cllp7

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-cllp7

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

multus

authorino-57f7f8659f-cllp7

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-f8vvn

kuadrant-system

multus

authorino-56c6c9ddd5-xvz6f

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

kuadrant-system

multus

limitador-limitador-7c9b87d574-f8vvn

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-56c6c9ddd5-xvz6f

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-xvz6f

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-f8vvn

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ff8bx

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.29s (2.29s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ff8bx

Started

Container started

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ff8bx

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-f8vvn

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-cllp7

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-cllp7

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-cllp7

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.434s (3.434s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-xvz6f

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-xvz6f

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-xvz6f

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.018s (3.018s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-f8vvn

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.825s (2.825s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-f8vvn

Started

Container started

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-cllp7

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

authorino-57f7f8659f-cllp7

Killing

Stopping container authorino

opendatahub

kserve-module-controller-manager-7566cf5fc8-qdflj_9e94dd55-f60a-49f6-ae6f-f5d2a64e42c5

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-qdflj_9e94dd55-f60a-49f6-ae6f-f5d2a64e42c5 became leader

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-ff8bx

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-ff8bx

Killing

Stopping container limitador
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kubelet

maas-controller-64678f5d5d-96mcs

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e"

opendatahub

replicaset-controller

maas-controller-64678f5d5d

SuccessfulCreate

Created pod: maas-controller-64678f5d5d-96mcs

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-64678f5d5d from 0 to 1

opendatahub

replicaset-controller

maas-controller-6dfdc587c5

SuccessfulCreate

Created pod: maas-controller-6dfdc587c5-jcn8k

opendatahub

multus

maas-controller-64678f5d5d-96mcs

AddedInterface

Add eth0 [10.134.0.25/23] from ovn-kubernetes

opendatahub

multus

maas-controller-6dfdc587c5-jcn8k

AddedInterface

Add eth0 [10.134.0.26/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-6dfdc587c5-jcn8k

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e"

opendatahub

kubelet

maas-controller-64678f5d5d-96mcs

Created

Container created

opendatahub

kubelet

maas-controller-64678f5d5d-96mcs

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e" in 2.99s (2.99s including waiting). Image size: 200107538 bytes.

opendatahub

kubelet

maas-controller-64678f5d5d-96mcs

Started

Container started

opendatahub

kubelet

maas-controller-6dfdc587c5-jcn8k

Started

Container started

opendatahub

kubelet

maas-controller-6dfdc587c5-jcn8k

Created

Container created

opendatahub

kubelet

maas-controller-6dfdc587c5-jcn8k

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e" in 2.8s (2.8s including waiting). Image size: 200107538 bytes.

opendatahub

maas-controller-64678f5d5d-96mcs_917656ea-aca4-4c0a-9522-54e0a02f7e76

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-64678f5d5d-96mcs_917656ea-aca4-4c0a-9522-54e0a02f7e76 became leader

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-m6wvl

AddedInterface

Add eth0 [10.134.0.27/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-m6wvl

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-kjghr

opendatahub

kubelet

kserve-controller-manager-776cc66d47-4ggmd

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-m6wvl

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-m6wvl

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-m6wvl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-75cwz

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-4ggmd

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-nmtcb

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-75cwz

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-75cwz

opendatahub

multus

model-serving-api-75bd89fdfc-78fsn

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-78fsn

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

multus

kserve-controller-manager-776cc66d47-4ggmd

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

opendatahub

kubelet

odh-model-controller-769875db56-nmtcb

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-78fsn

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

multus

odh-model-controller-769875db56-nmtcb

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-kjghr

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-kjghr

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

opendatahub

kubelet

model-serving-api-75bd89fdfc-78fsn

Started

Container started

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-mf4wk

opendatahub

kubelet

model-serving-api-75bd89fdfc-78fsn

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-78fsn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 1.936s (1.936s including waiting). Image size: 184925761 bytes.

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-mf4wk

Killing

Stopping container discovery

opendatahub

kubelet

kserve-controller-manager-776cc66d47-4ggmd

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-75cwz

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-75cwz

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-75cwz

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 3.025s (3.025s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-4ggmd

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.149s (3.149s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

kserve-controller-manager-776cc66d47-4ggmd

Created

Container created

opendatahub

odh-model-controller-769875db56-nmtcb_e002f85e-39c8-4987-9e4b-43300985eda3

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-nmtcb_e002f85e-39c8-4987-9e4b-43300985eda3 became leader

opendatahub

kubelet

odh-model-controller-769875db56-nmtcb

Started

Container started

opendatahub

kubelet

odh-model-controller-769875db56-nmtcb

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-nmtcb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.464s (3.464s including waiting). Image size: 236755567 bytes.

opendatahub

kserve-controller-manager-776cc66d47-4ggmd_6df18e95-d847-4244-8aca-5ce4b917e34e

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-4ggmd_6df18e95-d847-4244-8aca-5ce4b917e34e became leader

opendatahub

llmisvc-controller-manager-869d5dcc5d-75cwz_043d1b61-c4bc-4267-b218-3e11f9f52779

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-75cwz_043d1b61-c4bc-4267-b218-3e11f9f52779 became leader

opendatahub

replicaset-controller

maas-controller-64678f5d5d

SuccessfulDelete

Deleted pod: maas-controller-64678f5d5d-96mcs

opendatahub

kubelet

maas-controller-64678f5d5d-96mcs

Killing

Stopping container manager

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-64678f5d5d from 1 to 0

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-kjghr

Created

Container created

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-kjghr

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-kjghr

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 10.962s (10.962s including waiting). Image size: 1089502687 bytes.

opendatahub

maas-controller-6dfdc587c5-jcn8k_991cb41b-1a44-4e63-b27b-ef9af1179180

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-6dfdc587c5-jcn8k_991cb41b-1a44-4e63-b27b-ef9af1179180 became leader
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-lrwvj

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6889cbd8c4 from 1 to 0

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-lrwvj

kuadrant-system

replicaset-controller

authorino-5ffb77fcd4

SuccessfulCreate

Created pod: authorino-5ffb77fcd4-8kmbf

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulCreate

Created pod: authorino-6889cbd8c4-2p2jc

kuadrant-system

replicaset-controller

authorino-6889cbd8c4

SuccessfulDelete

Deleted pod: authorino-6889cbd8c4-2p2jc

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6889cbd8c4 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-5ffb77fcd4 from 0 to 1

kuadrant-system

kubelet

authorino-5ffb77fcd4-8kmbf

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-5ffb77fcd4-8kmbf

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Started

Container started

kuadrant-system

kubelet

authorino-5ffb77fcd4-8kmbf

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.219s (1.219s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-5ffb77fcd4-8kmbf

Created

Container created

kuadrant-system

kubelet

authorino-5ffb77fcd4-8kmbf

Started

Container started

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-xvz6f

kuadrant-system

kubelet

authorino-56c6c9ddd5-xvz6f

Killing

Stopping container authorino

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Created

Container created
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

ProbeError

Startup probe error: Get "http://10.133.0.17:15021/healthz/ready": dial tcp 10.133.0.17:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Unhealthy

Startup probe failed: Get "http://10.133.0.17:15021/healthz/ready": dial tcp 10.133.0.17:15021: connect: connection refused

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-54d5w

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

multus

llmisvc-controller-manager-6466c554d8-54d5w

AddedInterface

Add eth0 [10.132.0.47/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-54d5w

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-54d5w

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 363ms (363ms including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-54d5w

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-54d5w

Started

Container started
(x6)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-75cwz

Killing

Stopping container manager

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-75cwz

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5d8dddb676 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulCreate

Created pod: ai-gateway-controller-5d8dddb676-wzdx8

opendatahub

multus

ai-gateway-controller-5d8dddb676-wzdx8

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-wzdx8

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-wzdx8

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-wzdx8

Created

Container created

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-wzdx8

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 1.984s (1.984s including waiting). Image size: 183979430 bytes.

opendatahub

ai-gateway-controller-5d8dddb676-wzdx8_ac449f04-32e7-4df2-9d5a-470dfff9d3f6

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5d8dddb676-wzdx8_ac449f04-32e7-4df2-9d5a-470dfff9d3f6 became leader
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

FailedCreate

Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-b9jkn

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-r9bfq

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-55fbbbf9dc-r9bfq

AddedInterface

Add eth0 [10.134.0.32/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-7b98bb678c-b9jkn

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-b9jkn

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulCreate

Created pod: payload-processing-55fbbbf9dc-r9bfq

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-b9jkn

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.366s (2.366s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-r9bfq

Created

Container created

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-r9bfq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.239s (2.239s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-r9bfq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-b9jkn

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-b9jkn

Started

Container started

opendatahub

llmisvc-controller-manager-6466c554d8-54d5w_caa280a9-aff5-488e-9c52-4bfbdafa1ce8

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-54d5w_caa280a9-aff5-488e-9c52-4bfbdafa1ce8 became leader

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-5f464594c6 from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-5f464594c6

SuccessfulCreate

Created pod: ai-gateway-controller-5f464594c6-562pp

opendatahub

kubelet

ai-gateway-controller-5f464594c6-562pp

Created

Container created

opendatahub

kubelet

ai-gateway-controller-5f464594c6-562pp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 337ms (337ms including waiting). Image size: 183979430 bytes.

opendatahub

kubelet

ai-gateway-controller-5f464594c6-562pp

Started

Container started

opendatahub

kubelet

ai-gateway-controller-5f464594c6-562pp

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

multus

ai-gateway-controller-5f464594c6-562pp

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-5d8dddb676 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-5d8dddb676-wzdx8

Killing

Stopping container manager

opendatahub

replicaset-controller

ai-gateway-controller-5d8dddb676

SuccessfulDelete

Deleted pod: ai-gateway-controller-5d8dddb676-wzdx8

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.324s (1.324s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

Started

Container started

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-5qfl4

Created

Container created

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-d77tr

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-d77tr

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-d77tr

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-d77tr

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 222ms (222ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-d77tr

Created

Container created

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-d77tr

Started

Container started

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-tr28v

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-tr28v

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

Started

Container started

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

Created

Container created

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pbhb4

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.266s (1.266s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-tr28v

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-tr28v

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 196ms (196ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-tr28v

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-tr28v

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

Started

Container started

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

Created

Container created

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 275ms (275ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8b6c6n

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-c242s

llm

multus

echo-simulator-kserve-64b6f8f96c-c242s

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

kubelet

echo-simulator-kserve-64b6f8f96c-c242s

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

opendatahub

ai-gateway-controller-5f464594c6-562pp_1d82bb72-8b88-4491-82ff-d954ff6bd6cf

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-5f464594c6-562pp_1d82bb72-8b88-4491-82ff-d954ff6bd6cf became leader

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 604ms (604ms including waiting). Image size: 98346788 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-c242s

Started

Container started

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-vvxk9

Started

Container started

llm

kubelet

echo-simulator-kserve-64b6f8f96c-c242s

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 1.906s (1.906s including waiting). Image size: 78208878 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-c242s

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Started

Container started

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 197ms (197ms including waiting). Image size: 98346788 bytes.

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-92rg7

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 1 to 2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

kuadrant-system

replicaset-controller

authorino-948677c5f

SuccessfulCreate

Created pod: authorino-948677c5f-lc4kc

kuadrant-system

multus

authorino-948677c5f-lc4kc

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-948677c5f-lc4kc

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

replicaset-controller

authorino-5ffb77fcd4

SuccessfulDelete

Deleted pod: authorino-5ffb77fcd4-8kmbf

kuadrant-system

kubelet

authorino-948677c5f-lc4kc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 827ms (827ms including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-948677c5f-lc4kc

Created

Container created

kuadrant-system

kubelet

authorino-948677c5f-lc4kc

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.668s (2.668s including waiting). Image size: 183555333 bytes.

kuadrant-system

kubelet

authorino-5ffb77fcd4-8kmbf

Killing

Stopping container authorino
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-5ffb77fcd4 from 1 to 0
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Unhealthy

Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

ProbeError

Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 2 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

AddedInterface

Add eth0 [10.132.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.601s (2.601s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Created

Container created
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

ProbeError

Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Unhealthy

Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-z7p28

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 4 to 8

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

AddedInterface

Add eth0 [10.132.0.49/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

AddedInterface

Add eth0 [10.134.0.40/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-86c2e2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-86c2e2

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-156ae1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-156ae1

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 0 to 1

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-praxis-a-4429aa-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-4429aa-openshift-default-856555d6d5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-praxis-a-4429aa-openshift-default-856555d6d5

SuccessfulCreate

Created pod: e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

openshift-ingress

gateway_labeler_controller

e2e-praxis-a-4429aa

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-a-4429aa

openshift-ingress

multus

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

Unhealthy

Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

ProbeError

Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Unhealthy

Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

ProbeError

Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": dial tcp 10.133.0.41:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

ProbeError

Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Unhealthy

Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Unhealthy

Startup probe failed: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

ProbeError

Startup probe error: Get "http://10.133.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-156ae1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-156ae1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-156ae1: serviceaccount "payload-processing-e2e-worker-w3-156ae1" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-156ae1: serviceaccount "payload-processing-e2e-worker-w3-156ae1" not found

openshift-ingress

multus

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

AddedInterface

Add eth0 [10.133.0.44/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-86c2e2: serviceaccount "payload-processing-e2e-worker-w0-86c2e2" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-86c2e2: serviceaccount "payload-processing-e2e-worker-w0-86c2e2" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-86c2e2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-86c2e2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-a-4429aa

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-4429aa-7d69cbff69

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-a-4429aa-7d69cbff69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-4429aa: serviceaccount "payload-processing-e2e-praxis-a-4429aa" not found

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-a-4429aa

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-a-4429aa-7d69cbff69 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-4429aa: serviceaccount "payload-processing-e2e-praxis-a-4429aa" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-4429aa-7d69cbff69

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 331ms (331ms including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Created

Container created

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 457ms (457ms including waiting). Image size: 125353963 bytes.

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 9; reason:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 8 to 9

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.18s (2.18s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-4429aa-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-4429aa-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

Started

Container started

openshift-ingress

deployment-controller

e2e-praxis-b-0a0ec6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-b-0a0ec6-openshift-default-c4dff6869 from 0 to 1

openshift-ingress

replicaset-controller

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869

SuccessfulCreate

Created pod: e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

openshift-ingress

gateway_labeler_controller

e2e-praxis-b-0a0ec6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-b-0a0ec6
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Unhealthy

Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused

openshift-ingress

multus

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

AddedInterface

Add eth0 [10.133.0.51/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

ProbeError

Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

ProbeError

Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

Unhealthy

Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-0a0ec6: serviceaccount "payload-processing-e2e-praxis-b-0a0ec6" not found

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-b-0a0ec6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-b-0a0ec6-8595fdb499 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-b-0a0ec6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-0a0ec6: serviceaccount "payload-processing-e2e-praxis-b-0a0ec6" not found

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Unhealthy

Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

ProbeError

Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused body:

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 9 to 10

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

AddedInterface

Add eth0 [10.133.0.54/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-4429aa-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-4429aa-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Unhealthy

Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

ProbeError

Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc

SuccessfulCreate

Created pod: e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

openshift-ingress

multus

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

AddedInterface

Add eth0 [10.132.0.51/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w4-1ffaa0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-1ffaa0

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

ProbeError

Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-shared-a-w4-1ffaa0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Unhealthy

Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w4-1ffaa0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w4-1ffaa0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

AddedInterface

Add eth0 [10.134.0.50/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

Started

Container started
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-1ffaa0: serviceaccount "payload-processing-e2e-shared-a-w4-1ffaa0" not found
(x2)

openshift-ingress

kubelet

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

Created

Container created

openshift-ingress

multus

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-1ffaa0: serviceaccount "payload-processing-e2e-shared-a-w4-1ffaa0" not found
(x2)

openshift-ingress

kubelet

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

Started

Container started

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-0a0ec6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-0a0ec6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 1 to 2
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz
(x5)

openshift-ingress

kubelet

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

Unhealthy

Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

ProbeError

Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused body:

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

ProbeError

Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Unhealthy

Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w4-2d0da2

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-2d0da2
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

Started

Container started

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 0 to 1
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

Created

Container created

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

AddedInterface

Add eth0 [10.133.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-2d0da2: serviceaccount "payload-processing-e2e-shared-b-w4-2d0da2" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

Unhealthy

Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-2d0da2: serviceaccount "payload-processing-e2e-shared-b-w4-2d0da2" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w4-2d0da2

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w4-2d0da2

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f from 0 to 1
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

ProbeError

Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

AddedInterface

Add eth0 [10.133.0.57/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

AddedInterface

Add eth0 [10.133.0.58/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-0a0ec6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-0a0ec6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

AddedInterface

Add eth0 [10.134.0.55/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 2 to 4

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x7)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

ProbeError

Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Unhealthy

Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

ProbeError

Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Unhealthy

Readiness probe failed: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

ProbeError

Readiness probe error: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Unhealthy

Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

ProbeError

Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Unhealthy

Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

ProbeError

Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Unhealthy

Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

ProbeError

Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Unhealthy

Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Unhealthy

Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

ProbeError

Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused body:

opendatahub

multus

test-curl-7316-4069cb

AddedInterface

Add eth0 [10.132.0.57/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7316-4069cb

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

kubelet

test-curl-7316-4069cb

Started

Container started

opendatahub

kubelet

test-curl-7316-4069cb

Created

Container created

opendatahub

kubelet

test-curl-7316-4069cb

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.371s (2.371s including waiting). Image size: 106100986 bytes.

opendatahub

multus

test-curl-7316-83dcde

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7316-83dcde

Created

Container created

opendatahub

kubelet

test-curl-7316-83dcde

Started

Container started

opendatahub

kubelet

test-curl-7316-83dcde

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7316-4069cb

Killing

Stopping container test-curl-7316-4069cb

openshift-ingress

deployment-controller

e2e-shared-a-w4-1ffaa0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc

SuccessfulCreate

Created pod: e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

openshift-ingress

replicaset-controller

e2e-inf-a-5b97f1-openshift-default-7cff8c8879

SuccessfulCreate

Created pod: e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

openshift-ingress

multus

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

AddedInterface

Add eth0 [10.133.0.62/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7316-83dcde

Killing

Stopping container test-curl-7316-83dcde

openshift-ingress

multus

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-inf-a-5b97f1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-5b97f1

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

Started

Container started

openshift-ingress

deployment-controller

e2e-inf-a-5b97f1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-5b97f1-openshift-default-7cff8c8879 from 0 to 1

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

Created

Container created
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5b97f1: serviceaccount "payload-processing-e2e-inf-a-5b97f1" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-5b97f1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-5b97f1-56995974c5 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-5b97f1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-5b97f1-7954c5f865 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-5b97f1-7954c5f865

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-5b97f1-7954c5f865

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-5b97f1-7954c5f865-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-5b97f1: serviceaccount "payload-processing-e2e-inf-a-5b97f1" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

Started

Container started

openshift-ingress

multus

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

AddedInterface

Add eth0 [10.134.0.57/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

Unhealthy

Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

ProbeError

Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 2 to 4

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 4 to 5

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Started

Container started
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Unhealthy

Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

ProbeError

Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

ProbeError

Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Unhealthy

Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

Killing

Stopping container payload-pre-processing

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

AddedInterface

Add eth0 [10.134.0.61/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

AddedInterface

Add eth0 [10.133.0.66/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

AddedInterface

Add eth0 [10.133.0.65/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 5 to 10

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

Unhealthy

Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Unhealthy

Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Unhealthy

Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

ProbeError

Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

ProbeError

Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

Unhealthy

Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Unhealthy

Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

ProbeError

Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

ProbeError

Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

ProbeError

Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-4429aa-648f6bdb7-dhpmd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

e2e-inf-b-e759e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-e759e5-openshift-default-765cc7f4f from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-b-e759e5-openshift-default-765cc7f4f

SuccessfulCreate

Created pod: e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

openshift-ingress

gateway_labeler_controller

e2e-inf-b-e759e5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-e759e5
(x2)

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-4429aa-7d69cbff69-vvx2b

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

multus

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

AddedInterface

Add eth0 [10.132.0.66/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

ProbeError

Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

Unhealthy

Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-e759e5-57578fb859

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-e759e5-57578fb859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e759e5: serviceaccount "payload-processing-e2e-inf-b-e759e5" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-e759e5-57578fb859

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-e759e5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-e759e5: serviceaccount "payload-processing-e2e-inf-b-e759e5" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-e759e5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-e759e5-57578fb859 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

AddedInterface

Add eth0 [10.134.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

AddedInterface

Add eth0 [10.133.0.68/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

Created

Container created

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 4 to 8

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

AddedInterface

Add eth0 [10.133.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

AddedInterface

Add eth0 [10.134.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Started

Container started

openshift-ingress

kubelet

e2e-praxis-a-4429aa-openshift-default-856555d6d5-8g87g

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Unhealthy

Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

ProbeError

Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

ProbeError

Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

ProbeError

Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Unhealthy

Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Unhealthy

Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

ProbeError

Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

ProbeError

Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Unhealthy

Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-0a0ec6-8595fdb499-pvwkk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

Killing

Stopping container payload-pre-processing
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-0a0ec6-fcfcfc679-nfbhm

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.48:50052" within 1s: context deadline exceeded
(x2)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-praxis-b-0a0ec6-openshift-default-c4dff6869-p6qfs

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Unhealthy

Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

ProbeError

Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-extmod-a-331c9e-openshift-default-685b87bb69

SuccessfulCreate

Created pod: e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

openshift-ingress

deployment-controller

e2e-extmod-a-331c9e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-331c9e-openshift-default-685b87bb69 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-331c9e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-331c9e

openshift-ingress

multus

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

AddedInterface

Add eth0 [10.134.0.66/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-331c9e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-331c9e: serviceaccount "payload-processing-e2e-extmod-a-331c9e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-331c9e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-331c9e-b6dfd87cf from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-331c9e: serviceaccount "payload-processing-e2e-extmod-a-331c9e" not found

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

AddedInterface

Add eth0 [10.133.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Started

Container started

opendatahub

multus

test-curl-7322-8e57e7

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-8e57e7

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7322-8e57e7

Created

Container created

opendatahub

kubelet

test-curl-7322-8e57e7

Started

Container started

opendatahub

kubelet

test-curl-7322-8e57e7

Killing

Stopping container test-curl-7322-8e57e7
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-e759e5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-e759e5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 8 to 10

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

Created

Container created

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

AddedInterface

Add eth0 [10.134.0.69/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 1 to 2

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-inf-a-5b97f1-openshift-default-7cff8c8879

SuccessfulCreate

Created pod: e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-inf-a-5b97f1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-5b97f1-openshift-default-7cff8c8879 from 1 to 2

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

Started

Container started

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

Created

Container created

openshift-ingress

multus

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

AddedInterface

Add eth0 [10.133.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

Unhealthy

Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

ProbeError

Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

ProbeError

Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Unhealthy

Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

ProbeError

Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

Unhealthy

Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-inf-b-e759e5-openshift-default-765cc7f4f

SuccessfulCreate

Created pod: e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

openshift-ingress

deployment-controller

e2e-inf-b-e759e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-e759e5-openshift-default-765cc7f4f from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-e759e5-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

Started

Container started

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

Created

Container created

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7322-a13e07

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

multus

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

AddedInterface

Add eth0 [10.133.0.83/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-a13e07

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7322-a13e07

Started

Container started

opendatahub

kubelet

test-curl-7322-a13e07

Created

Container created

opendatahub

kubelet

test-curl-7322-b87bb1

Started

Container started

opendatahub

kubelet

test-curl-7322-a13e07

Killing

Stopping container test-curl-7322-a13e07

opendatahub

kubelet

test-curl-7322-b87bb1

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7322-b87bb1

Created

Container created

opendatahub

multus

test-curl-7322-b87bb1

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-b4361f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-b4361f

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

AddedInterface

Add eth0 [10.132.0.76/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-b87bb1

Killing

Stopping container test-curl-7322-b87bb1

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

ProbeError

Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Unhealthy

Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

opendatahub

kubelet

test-curl-7322-03bdd6

Created

Container created

opendatahub

kubelet

test-curl-7322-03bdd6

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7322-03bdd6

Started

Container started

opendatahub

multus

test-curl-7322-03bdd6

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

Unhealthy

Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

ProbeError

Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7322-843fbc

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-843fbc

Created

Container created

opendatahub

kubelet

test-curl-7322-03bdd6

Killing

Stopping container test-curl-7322-03bdd6

opendatahub

kubelet

test-curl-7322-843fbc

Started

Container started

opendatahub

kubelet

test-curl-7322-843fbc

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-b4361f: serviceaccount "payload-processing-e2e-extmod-b-b4361f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-b4361f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-b4361f-b4fdcf8df from 0 to 1

opendatahub

kubelet

test-curl-7322-843fbc

Killing

Stopping container test-curl-7322-843fbc

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-b4361f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-b4361f: serviceaccount "payload-processing-e2e-extmod-b-b4361f" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

Created

Container created

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

AddedInterface

Add eth0 [10.133.0.87/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

AddedInterface

Add eth0 [10.133.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Unhealthy

Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

ProbeError

Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-inf-a-5b97f1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-5b97f1-openshift-default-7cff8c8879 from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-inf-a-5b97f1-openshift-default-7cff8c8879

SuccessfulCreate

Created pod: e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

Created

Container created

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Unhealthy

Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

ProbeError

Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-inf-a-5b97f1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-5b97f1-openshift-default-7cff8c8879 from 3 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

Started

Container started

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

Created

Container created

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

AddedInterface

Add eth0 [10.133.0.100/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-inf-a-5b97f1-openshift-default-7cff8c8879

SuccessfulCreate

Created pod: e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

Killing

Stopping container payload-processing

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-675z5

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-6d2jm

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-675z5

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-xrpdz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-8jh2s

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-55f85cf5d7 from 10 to 6
(x6)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

ProbeError

Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

Unhealthy

Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused
(x3)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Unhealthy

Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

ProbeError

Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-e759e5-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-inf-b-e759e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-e759e5-openshift-default-765cc7f4f from 2 to 3

openshift-ingress

replicaset-controller

e2e-inf-b-e759e5-openshift-default-765cc7f4f

SuccessfulCreate

Created pod: e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Created

Container created

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Started

Container started

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

AddedInterface

Add eth0 [10.134.0.73/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 1 to 2

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x6)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

ProbeError

Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Unhealthy

Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

AddedInterface

Add eth0 [10.133.0.102/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-92rg7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-92rg7

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-55f85cf5d7 from 6 to 5

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Unhealthy

Liveness probe failed: Get "https://10.133.0.38:8000/health": context deadline exceeded
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

ProbeError

Startup probe error: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

Unhealthy

Startup probe failed: Get "http://10.133.0.102:15021/healthz/ready": dial tcp 10.133.0.102:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-a-5b97f1-7954c5f865-hks4t

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-5b97f1-56995974c5-hnm59

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-lsdw8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-cwgt6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-l88r6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-a-5b97f1-openshift-default-7cff8c8879-46nz6

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-5b97f1-openshift-default

FailedUpdateStatus

horizontalpodautoscalers.autoscaling "e2e-inf-a-5b97f1-openshift-default" not found

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Created

Container created

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

AddedInterface

Add eth0 [10.134.0.76/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

ProbeError

Readiness probe error: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Unhealthy

Liveness probe failed: Get "https://10.133.0.38:8000/health": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

ProbeError

Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Unhealthy

Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Unhealthy

Readiness probe failed: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x8)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": context deadline exceeded
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

ProbeError

Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Unhealthy

Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-cncgp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-e759e5-6c98644d9f-sls9f

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-r9t9j

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-b-e759e5-57578fb859-jjjsr

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

AddedInterface

Add eth0 [10.133.0.110/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-l7nnw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 2 to 3

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.863s (2.863s including waiting). Image size: 185476477 bytes.

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

AddedInterface

Add eth0 [10.134.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

Started

Container started

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

Created

Container created

openshift-ingress

kubelet

e2e-inf-b-e759e5-openshift-default-765cc7f4f-d8482

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x8)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

ProbeError

Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

Unhealthy

Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-ait-cbd38938-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-cbd38938-openshift-default-55744f6757 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-cbd38938-openshift-default-55744f6757

SuccessfulCreate

Created pod: e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

openshift-ingress

gateway_labeler_controller

e2e-ait-cbd38938

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-cbd38938

openshift-ingress

multus

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

AddedInterface

Add eth0 [10.134.0.78/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-cbd38938-654b48cf79

SuccessfulCreate

Created pod: payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

openshift-ingress

deployment-controller

payload-processing-e2e-ait-cbd38938

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-cbd38938-654b48cf79 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-cbd38938

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-cbd38938-5db46956c from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-cbd38938-654b48cf79

FailedCreate

Error creating: pods "payload-processing-e2e-ait-cbd38938-654b48cf79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-cbd38938: serviceaccount "payload-processing-e2e-ait-cbd38938" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-cbd38938-5db46956c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-cbd38938-5db46956c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-cbd38938-5db46956c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-cbd38938: serviceaccount "payload-processing-e2e-ait-cbd38938" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

AddedInterface

Add eth0 [10.134.0.80/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x3)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Unhealthy

Readiness probe failed: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

ProbeError

Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Unhealthy

Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

ProbeError

Readiness probe error: Get "http://10.132.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

ProbeError

Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Unhealthy

Readiness probe failed: Get "http://10.132.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Unhealthy

Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

ProbeError

Readiness probe error: Get "http://10.132.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Unhealthy

Readiness probe failed: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceNotReady

LLMInferenceService [premium-simulated-simulated-premium] is no longer Ready because of: MainWorkloadReady, WorkloadsReady

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

ProbeError

Readiness probe error: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-mt-4a67e25f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-4a67e25f

openshift-ingress

deployment-controller

e2e-mt-4a67e25f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-4a67e25f-openshift-default-7dcdb7754f from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-4a67e25f-openshift-default-7dcdb7754f

SuccessfulCreate

Created pod: e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

openshift-ingress

multus

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes
(x3)

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-4a67e25f-ffcd799c5

FailedCreate

Error creating: pods "payload-processing-e2e-mt-4a67e25f-ffcd799c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4a67e25f: serviceaccount "payload-processing-e2e-mt-4a67e25f" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-4a67e25f-ffcd799c5

SuccessfulCreate

Created pod: payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-4a67e25f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-4a67e25f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-4a67e25f-ffcd799c5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4a67e25f: serviceaccount "payload-processing-e2e-mt-4a67e25f" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

Started

Container started

openshift-ingress

multus

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

AddedInterface

Add eth0 [10.133.0.114/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

ProbeError

Readiness probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Unhealthy

Readiness probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 3 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

AddedInterface

Add eth0 [10.132.0.94/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Unhealthy

Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-cbd38938-5db46956c-mjgrw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-cbd38938-654b48cf79-mmknc

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

ProbeError

Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-4a67e25f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-4a67e25f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

ProbeError

Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

Unhealthy

Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-cbd38938-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-cbd38938-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-ait-cbd38938-openshift-default-55744f6757-mdddk

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-migrate-d608daf1-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-d608daf1-gw

openshift-ingress

deployment-controller

e2e-migrate-d608daf1-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf from 0 to 1

openshift-ingress

replicaset-controller

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf

SuccessfulCreate

Created pod: e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

openshift-ingress

multus

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 3 to 4

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

AddedInterface

Add eth0 [10.134.0.85/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-d608daf1-68cb77c885

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-d608daf1-68cb77c885

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-d608daf1-68cb77c885-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-d608daf1: serviceaccount "payload-processing-e2e-migrate-d608daf1" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-d608daf1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-d608daf1-68cb77c885 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-d608daf1: serviceaccount "payload-processing-e2e-migrate-d608daf1" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-d608daf1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-d608daf1-76c879f85f from 0 to 1

openshift-ingress

multus

payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh

AddedInterface

Add eth0 [10.134.0.87/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

ProbeError

Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Unhealthy

Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

ProbeError

Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Unhealthy

Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Unhealthy

Readiness probe failed: Get "http://10.132.0.92:15021/healthz/ready": read tcp 10.132.0.2:45306->10.132.0.92:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

ProbeError

Readiness probe error: Get "http://10.132.0.92:15021/healthz/ready": read tcp 10.132.0.2:45306->10.132.0.92:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

ProbeError

Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Unhealthy

Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-d608daf1-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-d608daf1-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 4 to 7

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

Started

Container started

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

AddedInterface

Add eth0 [10.133.0.127/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

Started

Container started

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

AddedInterface

Add eth0 [10.134.0.89/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

AddedInterface

Add eth0 [10.133.0.126/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-4a67e25f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-4a67e25f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x18)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-kt6ts

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
(x6)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

Unhealthy

Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

ProbeError

Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

Unhealthy

Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

ProbeError

Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

ProbeError

Startup probe error: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

Unhealthy

Startup probe failed: Get "http://10.133.0.127:15021/healthz/ready": dial tcp 10.133.0.127:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Started

Container started

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Unhealthy

Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

ProbeError

Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-4a67e25f-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

AddedInterface

Add eth0 [10.133.0.131/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-4a67e25f-openshift-default-7dcdb7754f

SuccessfulCreate

Created pod: e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

openshift-ingress

deployment-controller

e2e-mt-4a67e25f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-4a67e25f-openshift-default-7dcdb7754f from 1 to 2
(x6)

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Unhealthy

Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-migrate-d608daf1-68cb77c885-5l4kh

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-d608daf1-76c879f85f-msfmc

Killing

Stopping container payload-pre-processing
(x6)

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

ProbeError

Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

Created

Container created

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

Started

Container started

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-d608daf1-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-migrate-d608daf1-gw-openshift-default-cf68dbbf-kltrq

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-d608daf1-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

Unhealthy

Startup probe failed: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-ait-c1daa315

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c1daa315
(x6)

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

ProbeError

Startup probe error: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-ait-c1daa315-openshift-default-87b646887

SuccessfulCreate

Created pod: e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

openshift-ingress

deployment-controller

e2e-ait-c1daa315-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-c1daa315-openshift-default-87b646887 from 0 to 1

openshift-ingress

multus

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

AddedInterface

Add eth0 [10.134.0.91/23] from ovn-kubernetes

opendatahub

multus

test-curl-7319-1c0cee

AddedInterface

Add eth0 [10.132.0.105/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7319-1c0cee

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7319-1c0cee

Started

Container started

opendatahub

kubelet

test-curl-7319-1c0cee

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-c1daa315

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c1daa315: serviceaccount "payload-processing-e2e-ait-c1daa315" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-c1daa315-55d997c78f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-c1daa315-55d997c78f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c1daa315: serviceaccount "payload-processing-e2e-ait-c1daa315" not found

openshift-ingress

multus

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk

AddedInterface

Add eth0 [10.134.0.92/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-c1daa315

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-c1daa315-55d997c78f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-c1daa315-55d997c78f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

openshift-ingress

multus

payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

AddedInterface

Add eth0 [10.134.0.93/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7319-1c0cee

Killing

Stopping container test-curl-7319-1c0cee

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

Killing

Stopping container payload-pre-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c1daa315-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c1daa315-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 4 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

AddedInterface

Add eth0 [10.132.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Created

Container created

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

(combined from similar events): Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 7 to 10

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

AddedInterface

Add eth0 [10.134.0.95/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

Started

Container started

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

AddedInterface

Add eth0 [10.133.0.134/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

Created

Container created

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

AddedInterface

Add eth0 [10.132.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

Created

Container created
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-ss6wn

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

ProbeError

Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Unhealthy

Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

ProbeError

Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

ProbeError

Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-h9p4d

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

Unhealthy

Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

Unhealthy

Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

Unhealthy

Startup probe failed: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

ProbeError

Startup probe error: Get "http://10.132.0.108:15021/healthz/ready": dial tcp 10.132.0.108:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

ProbeError

Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

Unhealthy

Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-c1daa315-8689b9b6f5-qpnwk

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-c1daa315-55d997c78f-5hljh

Killing

Stopping container payload-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c1daa315-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-ait-c1daa315-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-c1daa315-openshift-default-87b646887 from 0 to 1
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-c1daa315-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-ait-c1daa315-openshift-default-87b646887

FailedCreate

Error creating: pods "e2e-ait-c1daa315-openshift-default-87b646887-" is forbidden: error looking up service account openshift-ingress/e2e-ait-c1daa315-openshift-default: serviceaccount "e2e-ait-c1daa315-openshift-default" not found

openshift-ingress

kubelet

e2e-ait-c1daa315-openshift-default-87b646887-fxnt9

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-09d577dc

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-09d577dc

openshift-ingress

multus

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

AddedInterface

Add eth0 [10.134.0.96/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ait-09d577dc-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-09d577dc-openshift-default-6cfd85bd75 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-09d577dc-openshift-default-6cfd85bd75

SuccessfulCreate

Created pod: e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Unhealthy

Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

ProbeError

Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

ProbeError

Readiness probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Unhealthy

Readiness probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-55f85cf5d7 from 5 to 3

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7lxxs

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-09d577dc: serviceaccount "payload-processing-e2e-ait-09d577dc" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

openshift-ingress

deployment-controller

payload-processing-e2e-ait-09d577dc

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-09d577dc-86ccb7d997 from 0 to 1

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-7lxxs
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-09d577dc-86ccb7d997

FailedCreate

Error creating: pods "payload-processing-e2e-ait-09d577dc-86ccb7d997-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-09d577dc: serviceaccount "payload-processing-e2e-ait-09d577dc" not found

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-09d577dc

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659 from 0 to 1

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lpzqg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

AddedInterface

Add eth0 [10.134.0.97/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-09d577dc-86ccb7d997

SuccessfulCreate

Created pod: payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

openshift-ingress

multus

payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-4a67e25f-ffcd799c5

SuccessfulCreate

Created pod: payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-4a67e25f-ffcd799c5

FailedCreate

Error creating: pods "payload-processing-e2e-mt-4a67e25f-ffcd799c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4a67e25f: serviceaccount "payload-processing-e2e-mt-4a67e25f" not found

openshift-ingress

kubelet

payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

openshift-ingress

deployment-controller

payload-processing-e2e-mt-4a67e25f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-4a67e25f-ffcd799c5 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

Started

Container started
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-4a67e25f: serviceaccount "payload-processing-e2e-mt-4a67e25f" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

Created

Container created

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-4a67e25f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

AddedInterface

Add eth0 [10.134.0.99/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq

AddedInterface

Add eth0 [10.133.0.138/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq

Started

Container started

openshift-ingress

multus

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

AddedInterface

Add eth0 [10.134.0.100/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-autores-6653eb

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-6653eb

openshift-ingress

deployment-controller

e2e-autores-6653eb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-6653eb-openshift-default-6df94d5785 from 0 to 1

openshift-ingress

replicaset-controller

e2e-autores-6653eb-openshift-default-6df94d5785

SuccessfulCreate

Created pod: e2e-autores-6653eb-openshift-default-6df94d5785-njjtz
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-6653eb-6d77d87d6c

FailedCreate

Error creating: pods "payload-processing-e2e-autores-6653eb-6d77d87d6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-6653eb: serviceaccount "payload-processing-e2e-autores-6653eb" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-6653eb

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-6653eb-5d84f9cb59 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-6653eb: serviceaccount "payload-processing-e2e-autores-6653eb" not found

openshift-ingress

deployment-controller

payload-processing-e2e-autores-6653eb

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-6653eb-6d77d87d6c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

openshift-ingress

kubelet

payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

Created

Container created

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-6653eb-6d77d87d6c

SuccessfulCreate

Created pod: payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

openshift-ingress

multus

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

AddedInterface

Add eth0 [10.133.0.139/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

ProbeError

Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Unhealthy

Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7319-2f122b

AddedInterface

Add eth0 [10.132.0.113/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7319-2f122b

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7319-2f122b

Created

Container created

opendatahub

kubelet

test-curl-7319-2f122b

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-09d577dc-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-09d577dc-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

opendatahub

kubelet

test-curl-7319-2f122b

Killing

Stopping container test-curl-7319-2f122b
(x6)

openshift-ingress

kubelet

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

Unhealthy

Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

ProbeError

Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 5 to 7

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-a-w4-1ffaa0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc from 2 to 3
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-09d577dc-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-09d577dc-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc

SuccessfulCreate

Created pod: e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

AddedInterface

Add eth0 [10.133.0.148/23] from ovn-kubernetes

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

AddedInterface

Add eth0 [10.132.0.114/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

Created

Container created

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-09d577dc-5cc6bb6659-5g2dr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-09d577dc-86ccb7d997-6d2sm

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

Started

Container started

openshift-ingress

multus

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

AddedInterface

Add eth0 [10.134.0.102/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

ProbeError

Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

Unhealthy

Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

ProbeError

Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

ProbeError

Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-ait-09d577dc-openshift-default-6cfd85bd75-8zwlw

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

Unhealthy

Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

Unhealthy

Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

Unhealthy

Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

ProbeError

Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-derive-95af53a1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-95af53a1-openshift-default-7f85f57d7c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-derive-95af53a1

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-95af53a1

openshift-ingress

replicaset-controller

e2e-derive-95af53a1-openshift-default-7f85f57d7c

SuccessfulCreate

Created pod: e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

openshift-ingress

multus

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

AddedInterface

Add eth0 [10.132.0.115/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-derive-95af53a1

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-95af53a1-86bd5fb577 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-95af53a1-86bd5fb577

FailedCreate

Error creating: pods "payload-processing-e2e-derive-95af53a1-86bd5fb577-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-95af53a1: serviceaccount "payload-processing-e2e-derive-95af53a1" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-95af53a1-86bd5fb577

SuccessfulCreate

Created pod: payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-95af53a1

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-95af53a1: serviceaccount "payload-processing-e2e-derive-95af53a1" not found

openshift-ingress

kubelet

payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

Started

Container started

openshift-ingress

multus

payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

AddedInterface

Add eth0 [10.134.0.105/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6

AddedInterface

Add eth0 [10.134.0.104/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-6653eb-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-6653eb-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-mt-4a67e25f-ffcd799c5-zv8hq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-4a67e25f-d9b789fc9-vgvbs

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

e2e-autores-6653eb-openshift-default-6df94d5785

SuccessfulCreate

Created pod: e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

openshift-ingress

deployment-controller

e2e-autores-6653eb-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-6653eb-openshift-default-6df94d5785 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-6653eb-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

Created

Container created

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

AddedInterface

Add eth0 [10.133.0.153/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

Started

Container started

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

AddedInterface

Add eth0 [10.134.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

(combined from similar events): Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w4-2d0da2-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs
(x5)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

Unhealthy

Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

ProbeError

Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-shared-b-w4-2d0da2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979 from 7 to 10

openshift-ingress

replicaset-controller

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979

SuccessfulCreate

Created pod: e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

AddedInterface

Add eth0 [10.133.0.154/23] from ovn-kubernetes

openshift-ingress

multus

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

AddedInterface

Add eth0 [10.132.0.117/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Created

Container created

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

Started

Container started

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-gj8jj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-4a67e25f-openshift-default-7dcdb7754f-kkk47

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

Unhealthy

Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

ProbeError

Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

Unhealthy

Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

ProbeError

Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Unhealthy

Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

ProbeError

Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Unhealthy

Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

ProbeError

Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-6653eb-5d84f9cb59-x2s5c

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-autores-6653eb-6d77d87d6c-mbh85

Killing

Stopping container payload-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-95af53a1-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-derive-95af53a1-86bd5fb577-gfnql

Killing

Stopping container payload-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-95af53a1-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-95af53a1-6ff6cd6d55-2xcv6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-api-a-cb587c-openshift-default-6f8dff86cc

SuccessfulCreate

Created pod: e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

openshift-ingress

gateway_labeler_controller

e2e-api-a-cb587c

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-cb587c

openshift-ingress

deployment-controller

e2e-api-a-cb587c-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-cb587c-openshift-default-6f8dff86cc from 0 to 1

openshift-ingress

multus

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

AddedInterface

Add eth0 [10.132.0.118/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-derive-95af53a1-openshift-default-7f85f57d7c-c2x5j

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-cb587c

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-cb587c

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-cb587c-749d8bd9d7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-cb587c-749d8bd9d7

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-cb587c: serviceaccount "payload-processing-e2e-api-a-cb587c" not found
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-cb587c-749d8bd9d7

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-cb587c-749d8bd9d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-cb587c: serviceaccount "payload-processing-e2e-api-a-cb587c" not found

openshift-ingress

kubelet

payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq

Created

Container created

openshift-ingress

multus

payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq

AddedInterface

Add eth0 [10.134.0.111/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

AddedInterface

Add eth0 [10.134.0.110/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-njjtz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-autores-6653eb-openshift-default-6df94d5785-fk9p5

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-explicit-4a0349-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-4a0349-openshift-default-5b8ff4f965 from 0 to 1

openshift-ingress

replicaset-controller

e2e-explicit-4a0349-openshift-default-5b8ff4f965

SuccessfulCreate

Created pod: e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

openshift-ingress

gateway_labeler_controller

e2e-explicit-4a0349

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-4a0349

openshift-ingress

multus

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

AddedInterface

Add eth0 [10.132.0.119/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-4a0349

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-4a0349-5dd598dcf9 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-4a0349-5dd598dcf9

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-4a0349-5dd598dcf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-4a0349: serviceaccount "payload-processing-e2e-explicit-4a0349" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-4a0349

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-4a0349-595768c78c from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-4a0349-595768c78c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-4a0349-595768c78c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-4a0349: serviceaccount "payload-processing-e2e-explicit-4a0349" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

AddedInterface

Add eth0 [10.134.0.112/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-4a0349-5dd598dcf9

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

openshift-ingress

multus

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

AddedInterface

Add eth0 [10.134.0.113/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-4a0349-595768c78c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

Started

Container started

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-z57r2

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

AddedInterface

Add eth0 [10.132.0.121/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

AddedInterface

Add eth0 [10.133.0.159/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

AddedInterface

Add eth0 [10.133.0.158/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-68ae22

Started

Container started

opendatahub

kubelet

test-curl-7322-68ae22

Created

Container created

opendatahub

kubelet

test-curl-7322-68ae22

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7322-68ae22

AddedInterface

Add eth0 [10.132.0.122/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-9e30f2

Started

Container started

opendatahub

kubelet

test-curl-7322-9e30f2

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7322-9e30f2

AddedInterface

Add eth0 [10.132.0.123/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7322-9e30f2

Created

Container created

opendatahub

kubelet

test-curl-7322-68ae22

Killing

Stopping container test-curl-7322-68ae22
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Unhealthy

Startup probe failed: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused

opendatahub

kubelet

test-curl-7322-9e30f2

Killing

Stopping container test-curl-7322-9e30f2
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

ProbeError

Startup probe error: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Unhealthy

Startup probe failed: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

ProbeError

Startup probe error: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

ProbeError

Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused body:
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Unhealthy

Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-wcq9z
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 6 to 10

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-rr46x

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-r88l9

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

AddedInterface

Add eth0 [10.134.0.115/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

AddedInterface

Add eth0 [10.133.0.161/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

AddedInterface

Add eth0 [10.132.0.124/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

AddedInterface

Add eth0 [10.132.0.125/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 10 to 6

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-4a0349-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulDelete

Deleted pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-s62tv

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulDelete

Deleted pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulDelete

Deleted pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-7chmr

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-8glj6

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulDelete

Deleted pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-b4b92

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-4a0349-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-api-b-2e8614-openshift-default-6784dcddf9

SuccessfulCreate

Created pod: e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

openshift-ingress

gateway_labeler_controller

e2e-api-b-2e8614

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-2e8614

openshift-ingress

deployment-controller

e2e-api-b-2e8614-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-2e8614-openshift-default-6784dcddf9 from 0 to 1
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Unhealthy

Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Unhealthy

Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

ProbeError

Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

ProbeError

Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Unhealthy

Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

ProbeError

Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

ProbeError

Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused body:

openshift-ingress

multus

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

AddedInterface

Add eth0 [10.134.0.116/23] from ovn-kubernetes
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Unhealthy

Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused

openshift-ingress

multus

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

AddedInterface

Add eth0 [10.134.0.117/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

ProbeError

Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Unhealthy

Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

Started

Container started

openshift-ingress

replicaset-controller

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc

SuccessfulCreate

Created pod: e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-a-w4-1ffaa0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc from 3 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Unhealthy

Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

ProbeError

Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-2e8614-6fbcdff689

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-2e8614

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-2e8614-6fbcdff689 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-2e8614-6fbcdff689

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-2e8614-6fbcdff689-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2e8614: serviceaccount "payload-processing-e2e-api-b-2e8614" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-2e8614

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-2e8614-589c8b7767 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-2e8614-589c8b7767

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-2e8614-589c8b7767

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-2e8614-589c8b7767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-2e8614: serviceaccount "payload-processing-e2e-api-b-2e8614" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn

AddedInterface

Add eth0 [10.133.0.164/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn

Started

Container started

openshift-ingress

multus

payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

AddedInterface

Add eth0 [10.133.0.165/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

Unhealthy

Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

ProbeError

Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

Unhealthy

Startup probe failed: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

ProbeError

Startup probe error: Get "http://10.134.0.117:15021/healthz/ready": dial tcp 10.134.0.117:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-cb587c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-cb587c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Unhealthy

Readiness probe failed: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

ProbeError

Readiness probe error: Get "http://10.132.0.119:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Unhealthy

Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

ProbeError

Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-cb587c-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-cb587c-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-2e8614-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-2e8614-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-65b496c9b from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-65b496c9b

SuccessfulCreate

Created pod: payload-processing-external-model-65b496c9b-pbkxk
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-65b496c9b

FailedCreate

Error creating: pods "payload-processing-external-model-65b496c9b-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-65b496c9b-pbkxk

Created

Container created

models-as-a-service

multus

payload-processing-external-model-65b496c9b-pbkxk

AddedInterface

Add eth0 [10.133.0.171/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-65b496c9b-pbkxk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-65b496c9b-pbkxk

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-4a0349-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-4a0349-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc

SuccessfulCreate

Created pod: e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

openshift-ingress

replicaset-controller

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc

SuccessfulCreate

Created pod: e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

openshift-ingress

deployment-controller

e2e-shared-a-w4-1ffaa0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc from 4 to 6

openshift-ingress

multus

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

AddedInterface

Add eth0 [10.133.0.172/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w4-1ffaa0-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Started

Container started

openshift-ingress

multus

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

AddedInterface

Add eth0 [10.132.0.127/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

Unhealthy

Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

ProbeError

Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Unhealthy

Startup probe failed: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

ProbeError

Startup probe error: Get "http://10.134.0.116:15021/healthz/ready": dial tcp 10.134.0.116:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

Unhealthy

Startup probe failed: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

ProbeError

Startup probe error: Get "http://10.132.0.127:15021/healthz/ready": dial tcp 10.132.0.127:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-api-a-cb587c-749d8bd9d7-7mrwq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-cb587c-5ccf5b4447-m4hn7

Killing

Stopping container payload-pre-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-64f7c4d6bf from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-64f7c4d6bf

SuccessfulCreate

Created pod: payload-processing-external-model-64f7c4d6bf-hddgq

models-as-a-service

multus

payload-processing-external-model-64f7c4d6bf-hddgq

AddedInterface

Add eth0 [10.133.0.175/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-64f7c4d6bf-hddgq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-64f7c4d6bf-hddgq

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-64f7c4d6bf-hddgq

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-2e8614-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-2e8614-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

Killing

Stopping container payload-pre-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-65b496c9b

SuccessfulDelete

Deleted pod: payload-processing-external-model-65b496c9b-pbkxk

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-65b496c9b from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-65b496c9b-pbkxk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-4a0349-595768c78c-6wfx8

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.112:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-processing-e2e-explicit-4a0349-5dd598dcf9-7fxc7

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.113:50052" within 1s: context deadline exceeded

models-as-a-service

multus

payload-processing-external-model-689fc7c65b-m8nsp

AddedInterface

Add eth0 [10.133.0.176/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-m8nsp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-api-a-cb587c-openshift-default-6f8dff86cc-vhlqc

Killing

Stopping container istio-proxy

models-as-a-service

replicaset-controller

payload-processing-external-model-689fc7c65b

SuccessfulCreate

Created pod: payload-processing-external-model-689fc7c65b-m8nsp

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-m8nsp

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-m8nsp

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-2e8614-589c8b7767-8wmxn

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-b-2e8614-6fbcdff689-kbdkq

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

e2e-noten-682a07-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-682a07-openshift-default-85774c99b9 from 0 to 1

openshift-ingress

replicaset-controller

e2e-noten-682a07-openshift-default-85774c99b9

SuccessfulCreate

Created pod: e2e-noten-682a07-openshift-default-85774c99b9-hrdms

openshift-ingress

kubelet

e2e-explicit-4a0349-openshift-default-5b8ff4f965-28xzf

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-noten-682a07

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-682a07

models-as-a-service

kubelet

payload-processing-external-model-64f7c4d6bf-hddgq

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-64f7c4d6bf

SuccessfulDelete

Deleted pod: payload-processing-external-model-64f7c4d6bf-hddgq

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-64f7c4d6bf from 1 to 0

openshift-ingress

kubelet

e2e-noten-682a07-openshift-default-85774c99b9-hrdms

Created

Container created

openshift-ingress

multus

e2e-noten-682a07-openshift-default-85774c99b9-hrdms

AddedInterface

Add eth0 [10.134.0.118/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-noten-682a07-openshift-default-85774c99b9-hrdms

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-noten-682a07-openshift-default-85774c99b9-hrdms

Started

Container started

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-b-2e8614-openshift-default-6784dcddf9-pwf5m

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-682a07-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-682a07-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

models-as-a-service

replicaset-controller

payload-processing-external-model-6d8799cd9

SuccessfulCreate

Created pod: payload-processing-external-model-6d8799cd9-tnpcj

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-6d8799cd9 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-6d8799cd9-tnpcj

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-6d8799cd9-tnpcj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-6d8799cd9-tnpcj

AddedInterface

Add eth0 [10.133.0.179/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6d8799cd9-tnpcj

Started

Container started

openshift-ingress

kubelet

e2e-noten-682a07-openshift-default-85774c99b9-hrdms

Killing

Stopping container istio-proxy

models-as-a-service

replicaset-controller

payload-processing-external-model-689fc7c65b

SuccessfulDelete

Deleted pod: payload-processing-external-model-689fc7c65b-m8nsp

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-m8nsp

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-689fc7c65b from 1 to 0

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

Started

Container started

openshift-ingress

deployment-controller

e2e-extmod-a-331c9e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-331c9e-openshift-default-685b87bb69 from 1 to 2

openshift-ingress

replicaset-controller

e2e-extmod-a-331c9e-openshift-default-685b87bb69

SuccessfulCreate

Created pod: e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

AddedInterface

Add eth0 [10.132.0.129/23] from ovn-kubernetes
(x8)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

Unhealthy

Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

ProbeError

Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulDelete

Deleted pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-smrcw

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 10 to 6

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulDelete

Deleted pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-twltq

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulDelete

Deleted pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dqn8q

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulDelete

Deleted pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-l4pj7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

models-as-a-service

replicaset-controller

payload-processing-external-model-689fc7c65b

SuccessfulCreate

Created pod: payload-processing-external-model-689fc7c65b-4wrkw
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-689fc7c65b from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-4wrkw

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-4wrkw

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-4wrkw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-689fc7c65b-4wrkw

AddedInterface

Add eth0 [10.134.0.119/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6d8799cd9-tnpcj

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-689fc7c65b-4wrkw

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-86c2e2-d85d5ffc5-wcgpl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-86c2e2-7b559c4579-j6z22

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w4-1ffaa0-7b44764f75-22szw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w4-1ffaa0-7db5bbf6b7-pjw88

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-45v77

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-x9tzt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-tkq28

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w0-86c2e2-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-86c2e2-openshift-default-6f7db89749 from 6 to 10
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-86c2e2-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

AddedInterface

Add eth0 [10.133.0.191/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Started

Container started

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

openshift-ingress

replicaset-controller

e2e-worker-w0-86c2e2-openshift-default-6f7db89749

SuccessfulCreate

Created pod: e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-jvqw6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-9bnvd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w4-1ffaa0-openshift-default-58876d79fc-69hsl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

AddedInterface

Add eth0 [10.133.0.192/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

AddedInterface

Add eth0 [10.133.0.190/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

AddedInterface

Add eth0 [10.134.0.120/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

ProbeError

Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Unhealthy

Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Unhealthy

Startup probe failed: Get "http://10.133.0.190:15021/healthz/ready": dial tcp 10.133.0.190:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

ProbeError

Startup probe error: Get "http://10.133.0.190:15021/healthz/ready": dial tcp 10.133.0.190:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Unhealthy

Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

ProbeError

Startup probe error: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Unhealthy

Startup probe failed: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

ProbeError

Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Unhealthy

Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

ProbeError

Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

ProbeError

Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Unhealthy

Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

ProbeError

Startup probe error: Get "http://10.133.0.191:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

ProbeError

Startup probe error: Get "http://10.133.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Unhealthy

Startup probe failed: Get "http://10.133.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Unhealthy

Startup probe failed: Get "http://10.133.0.191:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-2xzm4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-5shlg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-z7fw5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-gqxxc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-7vns4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-ssgqd

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-dn5m2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w0-86c2e2-openshift-default-dockercfg-7dv7n); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w0-86c2e2-openshift-default-dockercfg-7dv7n); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w0-86c2e2-openshift-default-dockercfg-7dv7n); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-74fq9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-brgls

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-86c2e2-openshift-default-6f7db89749-86w7s

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-extmod-a-331c9e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-331c9e-openshift-default-685b87bb69 from 2 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-a-331c9e-openshift-default-685b87bb69

SuccessfulCreate

Created pod: e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

AddedInterface

Add eth0 [10.133.0.194/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 8; reason: All metrics below target

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 10 to 8

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

Created

Container created

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulDelete

Deleted pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-5nzrg

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulDelete

Deleted pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-2zrh6

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x5)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

Unhealthy

Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-68ptk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-64pn9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-988gs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-29srw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lcsl4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-wmmsg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-lnlrs

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x5)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

ProbeError

Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-t5jm9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-765kf

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-shared-b-w4-2d0da2-openshift-default-7cc57b7979-vcvvm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w4-2d0da2-6d9558984f-bk4ds

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w4-2d0da2-7d857d58ff-xrj45

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-bf9485bb6

FailedCreate

Error creating: pods "payload-processing-external-model-bf9485bb6-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-bf9485bb6

SuccessfulCreate

Created pod: payload-processing-external-model-bf9485bb6-jrz9d

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-bf9485bb6 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-bf9485bb6-jrz9d

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-bf9485bb6-jrz9d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-bf9485bb6-jrz9d

Started

Container started

models-as-a-service

multus

payload-processing-external-model-bf9485bb6-jrz9d

AddedInterface

Add eth0 [10.133.0.195/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulDelete

Deleted pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulDelete

Deleted pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulDelete

Deleted pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-tdlcz

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 8 to 5

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-ss8x5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-7ds52

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 6 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulDelete

Deleted pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Unhealthy

Readiness probe failed: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

ProbeError

Readiness probe error: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Unhealthy

Readiness probe failed: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

Unhealthy

Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-tgkbb

ProbeError

Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

ProbeError

Readiness probe error: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

ProbeError

Readiness probe error: Get "http://10.132.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

ProbeError

Readiness probe error: Get "http://10.132.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Unhealthy

Readiness probe failed: Get "http://10.132.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Unhealthy

Readiness probe failed: Get "http://10.132.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulDelete

Deleted pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 5 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

ProbeError

Readiness probe error: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-bp5q8

Unhealthy

Readiness probe failed: Get "http://10.132.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

models-as-a-service

kubelet

payload-processing-external-model-bf9485bb6-jrz9d

Killing

Stopping container payload-processing
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-75cb49bc4f

FailedCreate

Error creating: pods "payload-processing-external-model-75cb49bc4f-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-75cb49bc4f from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-75cb49bc4f

SuccessfulCreate

Created pod: payload-processing-external-model-75cb49bc4f-gj7p4

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

Killing

Stopping container payload-pre-processing

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-gj7p4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-75cb49bc4f-gj7p4

AddedInterface

Add eth0 [10.134.0.123/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-gj7p4

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-gj7p4

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-gj7p4

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-75cb49bc4f from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-75cb49bc4f

SuccessfulCreate

Created pod: payload-processing-external-model-75cb49bc4f-m5hf9
(x7)

models-as-a-service

replicaset-controller

payload-processing-external-model-75cb49bc4f

FailedCreate

Error creating: pods "payload-processing-external-model-75cb49bc4f-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

multus

payload-processing-external-model-75cb49bc4f-m5hf9

AddedInterface

Add eth0 [10.134.0.124/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-m5hf9

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-m5hf9

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-m5hf9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-156ae1-6d45d5c8f9-g5rf5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-75cb49bc4f-m5hf9

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

openshift-ingress

replicaset-controller

e2e-worker-w3-156ae1-openshift-default-859f8b99df

SuccessfulCreate

Created pod: e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-156ae1-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w3-156ae1-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-156ae1-openshift-default-859f8b99df from 4 to 8

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

AddedInterface

Add eth0 [10.133.0.199/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

AddedInterface

Add eth0 [10.133.0.198/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

AddedInterface

Add eth0 [10.134.0.125/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

AddedInterface

Add eth0 [10.133.0.200/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

Created

Container created

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

Started

Container started

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

Started

Container started

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7cd8fc5c4 from 0 to 1
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-156ae1-7df59f7959-j7jdz

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-7cd8fc5c4

FailedCreate

Error creating: pods "payload-processing-external-model-7cd8fc5c4-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

kubelet

payload-processing-external-model-7cd8fc5c4-v9dth

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-7cd8fc5c4-v9dth

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-7cd8fc5c4-v9dth

Started

Container started

models-as-a-service

replicaset-controller

payload-processing-external-model-7cd8fc5c4

SuccessfulCreate

Created pod: payload-processing-external-model-7cd8fc5c4-v9dth

models-as-a-service

multus

payload-processing-external-model-7cd8fc5c4-v9dth

AddedInterface

Add eth0 [10.134.0.126/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7cd8fc5c4-v9dth

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

Unhealthy

Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

ProbeError

Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

Unhealthy

Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

ProbeError

Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

ProbeError

Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

ProbeError

Startup probe error: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

Unhealthy

Startup probe failed: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

Unhealthy

Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-wphv4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-djlxh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-thrxs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-c7vhc

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fjmmh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-lnc6z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-fkwwz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w3-156ae1-openshift-default-859f8b99df-2b94v

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 5 to 4

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulDelete

Deleted pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-brttl

openshift-ingress

deployment-controller

e2e-extmod-b-b4361f-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set e2e-extmod-b-b4361f-openshift-default-756f75c897 from 4 to 5

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-b4361f-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-extmod-b-b4361f-openshift-default-756f75c897

SuccessfulCreate

Created pod: e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

Started

Container started

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

Created

Container created

openshift-ingress

multus

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

AddedInterface

Add eth0 [10.134.0.137/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

Unhealthy

Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

ProbeError

Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused body:

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-r9t9j

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-r9t9j

Killing

Stopping container discovery

openshift-ingress

deployment-controller

e2e-extmod-a-331c9e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-331c9e-openshift-default-685b87bb69 from 3 to 4

openshift-ingress

replicaset-controller

e2e-extmod-a-331c9e-openshift-default-685b87bb69

SuccessfulCreate

Created pod: e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-331c9e-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

AddedInterface

Add eth0 [10.133.0.215/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

Created

Container created

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

Unhealthy

Startup probe failed: Get "http://10.133.0.215:15021/healthz/ready": dial tcp 10.133.0.215:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

ProbeError

Startup probe error: Get "http://10.133.0.215:15021/healthz/ready": dial tcp 10.133.0.215:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-l9css

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-cgtqt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-5l7w9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-a-331c9e-openshift-default-685b87bb69-9cnhr

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-331c9e-6d869d5455-fzvxd

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.133.0.76:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-331c9e-b6dfd87cf-j6wq2

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.133.0.77:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-wk9nb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-dpzhp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-bttp2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-b2bvz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-b4361f-openshift-default-756f75c897-qx4p4

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-rr46x

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-wcq9z
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wcq9z

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-qbxzw

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5h7r7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-rr46x

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-55f85cf5d7 from 10 to 5

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-8c02610e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8c02610e

openshift-ingress

replicaset-controller

e2e-ait-8c02610e-openshift-default-5f957ccdc9

SuccessfulCreate

Created pod: e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

openshift-ingress

multus

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

AddedInterface

Add eth0 [10.134.0.138/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-ait-8c02610e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-8c02610e-openshift-default-5f957ccdc9 from 0 to 1
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

Unhealthy

Readiness probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-bzkwg

ProbeError

Readiness probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-b4361f-59f6d9ddc9-5jffq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-b4361f-b4fdcf8df-h9vmf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x5)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8c02610e-6b5f48ffbf

FailedCreate

Error creating: pods "payload-processing-e2e-ait-8c02610e-6b5f48ffbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8c02610e: serviceaccount "payload-processing-e2e-ait-8c02610e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8c02610e-6b5f48ffbf

SuccessfulCreate

Created pod: payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

openshift-ingress

deployment-controller

payload-processing-e2e-ait-8c02610e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-8c02610e-6b5f48ffbf from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8c02610e: serviceaccount "payload-processing-e2e-ait-8c02610e" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-8c02610e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-8c02610e-69cddb4f5 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

AddedInterface

Add eth0 [10.133.0.220/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

AddedInterface

Add eth0 [10.133.0.219/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Unhealthy

Readiness probe failed: Get "http://10.134.0.138:15021/healthz/ready": read tcp 10.134.0.2:57786->10.134.0.138:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

ProbeError

Readiness probe error: Get "http://10.134.0.138:15021/healthz/ready": read tcp 10.134.0.2:57786->10.134.0.138:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Unhealthy

Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

ProbeError

Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8c02610e-69cddb4f5-pllpz

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-8c02610e-6b5f48ffbf-nhf8z

Killing

Stopping container payload-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8c02610e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8c02610e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Killing

Stopping container istio-proxy

opendatahub

kubelet

maas-controller-6dfdc587c5-jcn8k

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-6dfdc587c5

SuccessfulDelete

Deleted pod: maas-controller-6dfdc587c5-jcn8k

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-ait-8c02610e-openshift-default-5f957ccdc9-5sr9b

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

opendatahub

multus

maas-controller-6dfdc587c5-jl224

AddedInterface

Add eth0 [10.134.0.140/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-6dfdc587c5-jl224

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e"

opendatahub

replicaset-controller

maas-controller-6dfdc587c5

SuccessfulCreate

Created pod: maas-controller-6dfdc587c5-jl224

opendatahub

kubelet

maas-controller-6dfdc587c5-jl224

Created

Container created

opendatahub

kubelet

maas-controller-6dfdc587c5-jl224

Started

Container started

opendatahub

kubelet

maas-controller-6dfdc587c5-jl224

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e" in 454ms (454ms including waiting). Image size: 200107538 bytes.

opendatahub

maas-controller-6dfdc587c5-jl224_06a986eb-445c-4032-ba8c-498e5390c098

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-6dfdc587c5-jl224_06a986eb-445c-4032-ba8c-498e5390c098 became leader

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

AddedInterface

Add eth0 [10.134.0.141/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

AddedInterface

Add eth0 [10.132.0.135/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

AddedInterface

Add eth0 [10.132.0.136/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

AddedInterface

Add eth0 [10.133.0.223/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

AddedInterface

Add eth0 [10.132.0.134/23] from ovn-kubernetes

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 5 to 10

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Started

Container started

opendatahub

kubelet

maas-controller-6dfdc587c5-jl224

Killing

Stopping container manager
(x2)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-6dfdc587c5 from 1 to 0

opendatahub

replicaset-controller

maas-controller-6dfdc587c5

SuccessfulDelete

Deleted pod: maas-controller-6dfdc587c5-jl224
(x3)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-6dfdc587c5 from 0 to 1

opendatahub

replicaset-controller

maas-controller-6dfdc587c5

SuccessfulCreate

Created pod: maas-controller-6dfdc587c5-vbgqs

opendatahub

kubelet

maas-controller-6dfdc587c5-vbgqs

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e"

opendatahub

kubelet

maas-controller-6dfdc587c5-vbgqs

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a96550db54b9ae4f2a292b6ef077832c5b6099ebfae0902e5d8d24816e97dd9e" in 291ms (291ms including waiting). Image size: 200107538 bytes.

opendatahub

kubelet

maas-controller-6dfdc587c5-vbgqs

Created

Container created

opendatahub

kubelet

maas-controller-6dfdc587c5-vbgqs

Started

Container started

opendatahub

multus

maas-controller-6dfdc587c5-vbgqs

AddedInterface

Add eth0 [10.134.0.142/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

ProbeError

Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

ProbeError

Startup probe error: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Unhealthy

Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

ProbeError

Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Unhealthy

Startup probe failed: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Unhealthy

Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Unhealthy

Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": dial tcp 10.132.0.134:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Unhealthy

Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

ProbeError

Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

ProbeError

Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Unhealthy

Startup probe failed: Get "http://10.132.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Unhealthy

Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

ProbeError

Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Unhealthy

Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

ProbeError

Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

ProbeError

Startup probe error: Get "http://10.132.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

opendatahub

maas-controller-6dfdc587c5-vbgqs_6cecddd0-f870-4074-8f8d-64b9f8c8dc40

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-6dfdc587c5-vbgqs_6cecddd0-f870-4074-8f8d-64b9f8c8dc40 became leader

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.Deployment llm/e2e-prio-pool-ea8adfc6-kserve-router-scheduler

llm

replicaset-controller

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76

SuccessfulCreate

Created pod: e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.Secret llm/e2e-prio-pool-ea8adfc6-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.Service llm/e2e-prio-pool-ea8adfc6-epp-service

llm

replicaset-controller

e2e-prio-pool-ea8adfc6-kserve-7458dc747c

SuccessfulCreate

Created pod: e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.RoleBinding llm/e2e-prio-pool-ea8adfc6-epp-rb

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.Role llm/e2e-prio-pool-ea8adfc6-epp-role

llm

deployment-controller

e2e-prio-pool-ea8adfc6-kserve

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-ea8adfc6-kserve-7458dc747c from 0 to 1

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.ClusterRoleBinding /llm-e2e-prio-pool-ea8adfc6-epp-auth-rb

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.ServiceAccount llm/e2e-prio-pool-ea8adfc6-epp-sa

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.Service llm/e2e-prio-pool-ea8adfc6-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

Created v1.Deployment llm/e2e-prio-pool-ea8adfc6-kserve

llm

deployment-controller

e2e-prio-pool-ea8adfc6-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76 from 0 to 1

llm

multus

e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

AddedInterface

Add eth0 [10.133.0.228/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

Created

Container created

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Pulling

Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827"
(x9)

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Created

(combined from similar events): Created v1.DestinationRule llm/e2e-prio-pool-ea8adfc6-kserve-shadow-svc

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

Started

Container started

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Updated

Updated v1.HTTPRoute llm/e2e-prio-pool-ea8adfc6-kserve-route

llm

multus

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

AddedInterface

Add eth0 [10.133.0.229/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Created

Container created

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Started

Container started

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Pulled

Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827" in 1.661s (1.661s including waiting). Image size: 184206420 bytes.
(x2)

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Updated

Updated v1.Secret llm/e2e-prio-pool-ea8adfc6-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

LLMInferenceServiceReady

LLMInferenceService [e2e-prio-pool-ea8adfc6] is Ready

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Killing

Stopping container main

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-7458dc747c-gcb6d

Killing

Stopping container main

llm

LLMInferenceServiceController

e2e-prio-pool-ea8adfc6

Deleted

Deleted v1.ClusterRoleBinding /llm-e2e-prio-pool-ea8adfc6-epp-auth-rb

openshift-ingress

deployment-controller

e2e-worker-single-main-a58e3e-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-a58e3e-openshift-default-67d65d499c from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-a58e3e

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-a58e3e

openshift-ingress

multus

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

AddedInterface

Add eth0 [10.134.0.143/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c

SuccessfulCreate

Created pod: e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a58e3e: serviceaccount "payload-processing-e2e-worker-single-main-a58e3e" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-a58e3e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-a58e3e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a58e3e: serviceaccount "payload-processing-e2e-worker-single-main-a58e3e" not found

openshift-ingress

multus

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

AddedInterface

Add eth0 [10.134.0.146/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

AddedInterface

Add eth0 [10.134.0.145/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Created

Container created

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Unhealthy

Readiness probe failed: Get "http://10.134.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

ProbeError

Readiness probe error: Get "http://10.134.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Started

Container started
(x5)

llm

kubelet

e2e-prio-pool-ea8adfc6-kserve-router-scheduler-c697cb76-mtj4m

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

ProbeError

Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Unhealthy

Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-a58e3e-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-a58e3e-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5j5x6z

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.145:50052" within 1s: context deadline exceeded

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cpvktq

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.134.0.146:50052" within 1s: context deadline exceeded

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-a58e3e

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a58e3e: serviceaccount "payload-processing-e2e-worker-single-main-a58e3e" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5c855

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a58e3e: serviceaccount "payload-processing-e2e-worker-single-main-a58e3e" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-a58e3e

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-a58e3e-5b74b684c7 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

AddedInterface

Add eth0 [10.133.0.232/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj

AddedInterface

Add eth0 [10.133.0.231/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-a58e3e-5b74b684cmj42k

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-a58e3e-587c5vljrj

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-single-main-a58e3e-openshift-default-67d65d499c9vvql

Killing

Stopping container istio-proxy

llm

deployment-controller

e2e-same-id-a-0e21d4af-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-a-0e21d4af-kserve-7cd98c55f7 from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.Secret llm/e2e-same-id-a-0e21d4af-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.DestinationRule llm/e2e-same-id-a-0e21d4af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.Service llm/e2e-same-id-a-0e21d4af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.HTTPRoute llm/e2e-same-id-a-0e21d4af-kserve-route

llm

replicaset-controller

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7

SuccessfulCreate

Created pod: e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.Deployment llm/e2e-same-id-a-0e21d4af-kserve

llm

multus

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

AddedInterface

Add eth0 [10.133.0.234/23] from ovn-kubernetes

llm

kubelet

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

Started

Container started

llm

kubelet

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.NetworkPolicy llm/e2e-same-id-a-0e21d4af-prometheus-scraping

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.PodMonitor llm/e2e-same-id-a-0e21d4af-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Created

Created v1.PodMonitor llm/e2e-same-id-a-0e21d4af-kserve-llmisvc-engine-default

llm

kubelet

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

Created

Container created
(x2)

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

Updated

Updated v1.Secret llm/e2e-same-id-a-0e21d4af-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-a-0e21d4af] is Ready

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.DestinationRule llm/e2e-same-id-b-0e21d4af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.HTTPRoute llm/e2e-same-id-b-0e21d4af-kserve-route

llm

deployment-controller

e2e-same-id-b-0e21d4af-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-b-0e21d4af-kserve-6b856bff8f from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.Deployment llm/e2e-same-id-b-0e21d4af-kserve

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.Secret llm/e2e-same-id-b-0e21d4af-kserve-self-signed-certs

llm

replicaset-controller

e2e-same-id-b-0e21d4af-kserve-6b856bff8f

SuccessfulCreate

Created pod: e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.Service llm/e2e-same-id-b-0e21d4af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.PodMonitor llm/e2e-same-id-b-0e21d4af-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.PodMonitor llm/e2e-same-id-b-0e21d4af-kserve-llmisvc-engine-default

llm

kubelet

e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

Created

Container created

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Created

Created v1.NetworkPolicy llm/e2e-same-id-b-0e21d4af-prometheus-scraping

llm

kubelet

e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

Started

Container started

llm

multus

e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

AddedInterface

Add eth0 [10.133.0.235/23] from ovn-kubernetes
(x2)

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

Updated

Updated v1.Secret llm/e2e-same-id-b-0e21d4af-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-b-0e21d4af] is Ready
(x3)

llm

LLMInferenceServiceController

e2e-same-id-a-0e21d4af

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-b-0e21d4af in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-a-0e21d4af; safe to ignore if traffic splitting is planned
(x7)

llm

LLMInferenceServiceController

e2e-same-id-b-0e21d4af

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-a-0e21d4af in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-b-0e21d4af; safe to ignore if traffic splitting is planned

llm

kubelet

e2e-same-id-a-0e21d4af-kserve-7cd98c55f7-wx4td

Killing

Stopping container main

llm

kubelet

e2e-same-id-b-0e21d4af-kserve-6b856bff8f-phj2r

Killing

Stopping container main

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Unhealthy

Readiness probe failed: Get "http://10.133.0.223:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

ProbeError

Readiness probe error: Get "http://10.133.0.223:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Unhealthy

Readiness probe failed: Get "http://10.134.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

ProbeError

Readiness probe error: Get "http://10.134.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

e2e-ipp-a-72b663-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-72b663-openshift-default-86c96cd447 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-72b663

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-72b663

openshift-ingress

replicaset-controller

e2e-ipp-a-72b663-openshift-default-86c96cd447

SuccessfulCreate

Created pod: e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

openshift-ingress

multus

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

AddedInterface

Add eth0 [10.133.0.239/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-72b663

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-72b663-596b566c8f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-72b663-596b566c8f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

AddedInterface

Add eth0 [10.133.0.241/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-72b663-596b566c8f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-72b663-596b566c8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-72b663: serviceaccount "payload-processing-e2e-ipp-a-72b663" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-72b663

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-72b663: serviceaccount "payload-processing-e2e-ipp-a-72b663" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

AddedInterface

Add eth0 [10.133.0.240/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

Killing

Stopping container payload-processing
(x4)

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-656bdbd5ff-slc92

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-plugins-e2e-ipp-a-72b663" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-72b663

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-72b663-8665fcd66b from 0 to 1
(x4)

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-596b566c8f-vvw4p

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-plugins-e2e-ipp-a-72b663" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-72b663

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-72b663-7698d78955 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-72b663-7698d78955

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-72b663-7698d78955

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-72b663-7698d78955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-72b663: serviceaccount "payload-processing-e2e-ipp-a-72b663" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-72b663-8665fcd66b

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-72b663-8665fcd66b

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-72b663-8665fcd66b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-72b663: serviceaccount "payload-processing-e2e-ipp-a-72b663" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

AddedInterface

Add eth0 [10.134.0.149/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

AddedInterface

Add eth0 [10.133.0.242/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.358s (1.358s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.591s (1.591s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

Started

Container started

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

ProbeError

Readiness probe error: Get "http://10.133.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Unhealthy

Readiness probe failed: Get "http://10.133.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Unhealthy

Startup probe failed: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

ProbeError

Startup probe error: Get "http://10.133.0.239:15021/healthz/ready": dial tcp 10.133.0.239:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-72b663-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-72b663-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-131649

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-131649

openshift-ingress

deployment-controller

e2e-ipp-b-131649-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-131649-openshift-default-69b48db985 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ipp-b-131649-openshift-default-69b48db985

SuccessfulCreate

Created pod: e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

openshift-ingress

multus

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

AddedInterface

Add eth0 [10.133.0.243/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-131649

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-131649-5c9644669f

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-131649-5c9644669f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-131649: serviceaccount "payload-processing-e2e-ipp-b-131649" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-131649: serviceaccount "payload-processing-e2e-ipp-b-131649" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-131649

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-131649-5c9644669f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-131649-5c9644669f

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

AddedInterface

Add eth0 [10.134.0.151/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

AddedInterface

Add eth0 [10.134.0.150/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

Killing

Stopping container payload-processing
(x4)

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-5c9644669f-9jkkh

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-plugins-e2e-ipp-b-131649" not found
(x4)

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-5ff5bd9cb7-2gst9

FailedMount

MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-plugins-e2e-ipp-b-131649" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-131649

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-131649-58674d76d7 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-131649

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-131649-6f6c9596d4 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-131649-6f6c9596d4

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-131649-58674d76d7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-131649-58674d76d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-131649: serviceaccount "payload-processing-e2e-ipp-b-131649" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl

AddedInterface

Add eth0 [10.133.0.244/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-131649-58674d76d7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-131649-6f6c9596d4

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-131649-6f6c9596d4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-131649: serviceaccount "payload-processing-e2e-ipp-b-131649" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl

Created

Container created

openshift-ingress

multus

payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6

AddedInterface

Add eth0 [10.134.0.153/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Unhealthy

Startup probe failed: Get "http://10.133.0.243:15021/healthz/ready": dial tcp 10.133.0.243:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

ProbeError

Startup probe error: Get "http://10.133.0.243:15021/healthz/ready": dial tcp 10.133.0.243:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-72b663-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-72b663-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-131649-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-131649-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-131649-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-131649-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-72b663-8665fcd66b-z2bsj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-72b663-7698d78955-r4gg6

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-a-72b663-openshift-default-86c96cd447-4db8c

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-131649-58674d76d7-vjhhl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-131649-6f6c9596d4-wrjj6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-559f69b56 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-559f69b56-h5sc2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-559f69b56

SuccessfulCreate

Created pod: payload-pre-processing-559f69b56-h5sc2

openshift-ingress

multus

payload-pre-processing-559f69b56-h5sc2

AddedInterface

Add eth0 [10.134.0.154/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-559f69b56-h5sc2

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-559f69b56-h5sc2

Started

Container started

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ipp-b-131649-openshift-default-69b48db985-v7j9t

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulDelete

Deleted pod: payload-pre-processing-7b98bb678c-b9jkn

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-7b98bb678c from 1 to 0

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-b9jkn

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-5985fbdcbc from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-5985fbdcbc

SuccessfulCreate

Created pod: payload-processing-5985fbdcbc-h9tqm

openshift-ingress

kubelet

payload-processing-5985fbdcbc-h9tqm

Created

Container created

openshift-ingress

multus

payload-processing-5985fbdcbc-h9tqm

AddedInterface

Add eth0 [10.133.0.246/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-5985fbdcbc-h9tqm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-5985fbdcbc-h9tqm

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulDelete

Deleted pod: payload-processing-55fbbbf9dc-r9bfq

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-r9bfq

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-55fbbbf9dc from 1 to 0

openshift-ingress

replicaset-controller

payload-processing-95b747dcd

SuccessfulCreate

Created pod: payload-processing-95b747dcd-rw869

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-95b747dcd from 0 to 1

openshift-ingress

multus

payload-processing-95b747dcd-rw869

AddedInterface

Add eth0 [10.133.0.247/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-76bfcf8f7d from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-76bfcf8f7d

SuccessfulCreate

Created pod: payload-pre-processing-76bfcf8f7d-tzw9j

openshift-ingress

kubelet

payload-pre-processing-76bfcf8f7d-tzw9j

Started

Container started

openshift-ingress

kubelet

payload-processing-95b747dcd-rw869

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-95b747dcd-rw869

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-76bfcf8f7d-tzw9j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-76bfcf8f7d-tzw9j

AddedInterface

Add eth0 [10.134.0.155/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-76bfcf8f7d-tzw9j

Created

Container created

openshift-ingress

kubelet

payload-processing-95b747dcd-rw869

Started

Container started

openshift-ingress

kubelet

payload-processing-5985fbdcbc-h9tqm

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

payload-processing-5985fbdcbc

SuccessfulDelete

Deleted pod: payload-processing-5985fbdcbc-h9tqm

openshift-ingress

kubelet

payload-pre-processing-559f69b56-h5sc2

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-559f69b56

SuccessfulDelete

Deleted pod: payload-pre-processing-559f69b56-h5sc2

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-559f69b56 from 1 to 0

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-5985fbdcbc from 1 to 0

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5bcd7997 from 0 to 1
(x6)

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

FailedCreate

Error creating: pods "payload-processing-external-model-5bcd7997-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

SuccessfulCreate

Created pod: payload-processing-external-model-5bcd7997-l5wbj

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-l5wbj

Started

Container started

models-as-a-service

multus

payload-processing-external-model-5bcd7997-l5wbj

AddedInterface

Add eth0 [10.134.0.156/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-l5wbj

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-l5wbj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-l5wbj

Killing

Stopping container payload-processing
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

ProbeError

Readiness probe error: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Unhealthy

Readiness probe failed: Get "http://10.132.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

ProbeError

Readiness probe error: Get "http://10.132.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Unhealthy

Readiness probe failed: Get "http://10.132.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

ProbeError

Readiness probe error: Get "http://10.132.0.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Unhealthy

Readiness probe failed: Get "http://10.132.0.136:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x9)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Unhealthy

Readiness probe failed: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

ProbeError

Readiness probe error: Get "http://10.133.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Unhealthy

Readiness probe failed: Get "http://10.133.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

ProbeError

Readiness probe error: Get "http://10.133.0.158:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Unhealthy

Readiness probe failed: Get "http://10.133.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

ProbeError

Readiness probe error: Get "http://10.133.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Unhealthy

Readiness probe failed: Get "http://10.133.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x10)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

ProbeError

Readiness probe error: Get "http://10.134.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-ecd9e5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-ecd9e5

openshift-ingress

deployment-controller

e2e-worker-single-main-ecd9e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-ecd9e5-openshift-default-549cb596cc from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-single-main-ecd9e5-openshift-default-549cb596cc

SuccessfulCreate

Created pod: e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

openshift-ingress

multus

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

AddedInterface

Add eth0 [10.134.0.157/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ecd9e5: serviceaccount "payload-processing-e2e-worker-single-main-ecd9e5" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-ecd9e5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

AddedInterface

Add eth0 [10.133.1.7/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-ecd9e5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887 from 0 to 1
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ecd9e5: serviceaccount "payload-processing-e2e-worker-single-main-ecd9e5" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

AddedInterface

Add eth0 [10.133.1.8/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

ProbeError

Readiness probe error: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Unhealthy

Readiness probe failed: Get "http://10.134.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

ProbeError

Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Unhealthy

Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-ecd9e5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-ecd9e5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95k55zf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd4n55w

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

AddedInterface

Add eth0 [10.133.1.11/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-ecd9e5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ecd9e5: serviceaccount "payload-processing-e2e-worker-single-main-ecd9e5" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bd9

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ecd9e5: serviceaccount "payload-processing-e2e-worker-single-main-ecd9e5" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-ecd9e5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc954f887 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4

AddedInterface

Add eth0 [10.133.1.12/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-ecd9e5-5876c66bdx5jh4

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-ecd9e5-5bc95xwt8h

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-single-main-ecd9e5-openshift-default-549cb596ccclcp5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-82pkj

Killing

Stopping container manager

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-678d87766f from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-678d87766f-82pkj

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-5rnng
(x2)

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5rnng

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5rnng

Started

Container started
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-5rnng

Created

Container created

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-5rnng

AddedInterface

Add eth0 [10.132.0.142/23] from ovn-kubernetes
(x3)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x3)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-5rnng_ba3c0e01-84ba-4cb4-8e43-715140a85180

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-5rnng_ba3c0e01-84ba-4cb4-8e43-715140a85180 became leader

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-8r2q9

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7x8hv

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-z57r2

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-55f85cf5d7 from 10 to 1

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z7p28

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-55f85cf5d7-9qpkm

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z57r2

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-r88l9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-qpbjh

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-5btvs

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-mcrdq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-55f85cf5d7

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-55f85cf5d7-z6fd6
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 1 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

AddedInterface

Add eth0 [10.132.0.145/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

AddedInterface

Add eth0 [10.133.1.20/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

AddedInterface

Add eth0 [10.133.1.21/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

Created

Container created
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

ProbeError

Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

Unhealthy

Startup probe failed: Get "http://10.133.1.21:15021/healthz/ready": dial tcp 10.133.1.21:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

ProbeError

Startup probe error: Get "http://10.133.1.21:15021/healthz/ready": dial tcp 10.133.1.21:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

Unhealthy

Startup probe failed: Get "http://10.133.1.20:15021/healthz/ready": dial tcp 10.133.1.20:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

ProbeError

Startup probe error: Get "http://10.133.1.20:15021/healthz/ready": dial tcp 10.133.1.20:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

Unhealthy

Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 4 to 7

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 7; reason:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

AddedInterface

Add eth0 [10.134.0.161/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

AddedInterface

Add eth0 [10.133.1.22/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

AddedInterface

Add eth0 [10.132.0.146/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

Unhealthy

Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

ProbeError

Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

Unhealthy

Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

Unhealthy

Startup probe failed: Get "http://10.133.1.22:15021/healthz/ready": dial tcp 10.133.1.22:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

ProbeError

Startup probe error: Get "http://10.133.1.22:15021/healthz/ready": dial tcp 10.133.1.22:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

ProbeError

Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-55f85cf5d7 from 7 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

AddedInterface

Add eth0 [10.134.0.162/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

AddedInterface

Add eth0 [10.132.0.147/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

AddedInterface

Add eth0 [10.133.1.24/23] from ovn-kubernetes
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

ProbeError

Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

Unhealthy

Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

ProbeError

Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

Unhealthy

Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

ProbeError

Startup probe error: Get "http://10.133.1.24:15021/healthz/ready": dial tcp 10.133.1.24:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

Unhealthy

Startup probe failed: Get "http://10.133.1.24:15021/healthz/ready": dial tcp 10.133.1.24:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-mt-7b218f2f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-7b218f2f

openshift-ingress

deployment-controller

e2e-mt-7b218f2f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-7b218f2f-openshift-default-67d75c69f7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-7b218f2f-openshift-default-67d75c69f7

SuccessfulCreate

Created pod: e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

openshift-ingress

multus

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

AddedInterface

Add eth0 [10.133.1.25/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-7b218f2f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-7b218f2f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-7b218f2f-8485696fcf from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-7b218f2f-8485696fcf

SuccessfulCreate

Created pod: payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv
(x6)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-7b218f2f-8485696fcf

FailedCreate

Error creating: pods "payload-processing-e2e-mt-7b218f2f-8485696fcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7b218f2f: serviceaccount "payload-processing-e2e-mt-7b218f2f" not found
(x5)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7b218f2f: serviceaccount "payload-processing-e2e-mt-7b218f2f" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

openshift-ingress

kubelet

payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

AddedInterface

Add eth0 [10.134.0.164/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv

Created

Container created

openshift-ingress

multus

payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv

AddedInterface

Add eth0 [10.134.0.165/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

ProbeError

Readiness probe error: Get "http://10.133.1.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Unhealthy

Readiness probe failed: Get "http://10.133.1.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Unhealthy

Startup probe failed: Get "http://10.133.1.25:15021/healthz/ready": dial tcp 10.133.1.25:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

ProbeError

Startup probe error: Get "http://10.133.1.25:15021/healthz/ready": dial tcp 10.133.1.25:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-7b218f2f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-7b218f2f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-7b218f2f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-7b218f2f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-7b218f2f-66d74bf9b7-xtdpj

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-7b218f2f-8485696fcf-t5bhv

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-mt-7b218f2f-openshift-default-67d75c69f7-69r76

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

ProbeError

Readiness probe error: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-dzkt9

Unhealthy

Readiness probe failed: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

ProbeError

Readiness probe error: Get "http://10.133.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-wxgc4

Unhealthy

Readiness probe failed: Get "http://10.133.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

Unhealthy

Readiness probe failed: Get "http://10.132.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-t7kz9

ProbeError

Readiness probe error: Get "http://10.132.0.145:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

ProbeError

Readiness probe error: Get "http://10.133.1.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-c8dts

Unhealthy

Readiness probe failed: Get "http://10.133.1.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x11)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

Unhealthy

Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x11)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7jwxs

ProbeError

Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

Unhealthy

Readiness probe failed: Get "http://10.133.1.20:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

ProbeError

Readiness probe error: Get "http://10.133.1.21:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

Unhealthy

Readiness probe failed: Get "http://10.132.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

ProbeError

Readiness probe error: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-7g4cr

ProbeError

Readiness probe error: Get "http://10.132.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-z6fd6

ProbeError

Readiness probe error: Get "http://10.133.1.20:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

ProbeError

Readiness probe error: Get "http://10.132.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-2p5f4

Unhealthy

Readiness probe failed: Get "http://10.133.1.21:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-lnxjp

Unhealthy

Readiness probe failed: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-55f85cf5d7-cthxb

Unhealthy

Readiness probe failed: Get "http://10.132.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)