Time Namespace Component RelatedObject Reason Message

openshift-ingress

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-7ldgk to ip-10-0-140-68.us-west-1.compute.internal

ai-tenants

e2e-autores-a3e841

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-autores-a3e841: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-25719d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-25719d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-c79abe

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-c79abe: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-a-caf74d

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-a-caf74d: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-1ff1dc

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-1ff1dc: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-581487

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-581487: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-extmod-b-dc6e48

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-extmod-b-dc6e48: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-a-85584b

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-a-85584b: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-inf-b-ef9b55

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-inf-b-ef9b55: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-a-bd9d24

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-a-bd9d24: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-ipp-b-b354e3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-ipp-b-b354e3: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-47bc8f48

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-47bc8f48: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-mt-c831a4c0

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-mt-c831a4c0: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-a-e7b2a3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-a-e7b2a3: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-praxis-b-390a75

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-praxis-b-390a75: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w0-302ba0

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w0-302ba0: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w2-3db672

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w2-3db672: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-a-w3-2e0116

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-a-w3-2e0116: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w0-6ba6e5

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w0-6ba6e5: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w2-c52bfa

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w2-c52bfa: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-shared-b-w3-51aa8f

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-shared-b-w3-51aa8f: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-main-976689

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-main-976689: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w0-3ba2f3

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w0-3ba2f3: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w2-8ff537

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w2-8ff537: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w3-28c598

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w3-28c598: MaasTenantConfig is not available to restore revocation operands

ai-tenants

e2e-worker-w4-190724

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-worker-w4-190724: MaasTenantConfig is not available to restore revocation operands

cert-manager

cert-manager-597b95cb84-kbvqq

Scheduled

Successfully assigned cert-manager/cert-manager-597b95cb84-kbvqq to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Scheduled

Successfully assigned openshift-ingress/e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Scheduled

Successfully assigned openshift-ingress/e2e-ait-56ae52d5-openshift-default-796558d487-6gp99 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt to ip-10-0-140-68.us-west-1.compute.internal

cert-manager

cert-manager-cainjector-5c57bbd4fc-4g7gv

Scheduled

Successfully assigned cert-manager/cert-manager-cainjector-5c57bbd4fc-4g7gv to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Scheduled

Successfully assigned openshift-ingress/e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd

Scheduled

Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd to ip-10-0-140-68.us-west-1.compute.internal

cert-manager

cert-manager-webhook-5dbbbf75cc-rxfxj

Scheduled

Successfully assigned cert-manager/cert-manager-webhook-5dbbbf75cc-rxfxj to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng to ip-10-0-138-21.us-west-1.compute.internal

llm

echo-simulator-kserve-64b6f8f96c-5bs7b

Scheduled

Successfully assigned llm/echo-simulator-kserve-64b6f8f96c-5bs7b to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

Scheduled

Successfully assigned openshift-ingress/e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Scheduled

Successfully assigned openshift-ingress/e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz to ip-10-0-130-54.us-west-1.compute.internal

kuadrant-system

authorino-56c6c9ddd5-ggd22

Scheduled

Successfully assigned kuadrant-system/authorino-56c6c9ddd5-ggd22 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk to ip-10-0-138-21.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-5965b9bff7-zmv2g to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5 to ip-10-0-140-68.us-west-1.compute.internal

kuadrant-system

authorino-57f7f8659f-xjhwv

Scheduled

Successfully assigned kuadrant-system/authorino-57f7f8659f-xjhwv to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr to ip-10-0-130-54.us-west-1.compute.internal

llm

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

Scheduled

Successfully assigned llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg to ip-10-0-140-68.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-5965b9bff7-vr9xt to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

Scheduled

Successfully assigned openshift-ingress/e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89 to ip-10-0-138-21.us-west-1.compute.internal

kuadrant-system

authorino-58588b67c6-mr9nn

Scheduled

Successfully assigned kuadrant-system/authorino-58588b67c6-mr9nn to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

Scheduled

Successfully assigned openshift-ingress/e2e-worker-main-976689-openshift-default-5db554f479-nkwj5 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

Scheduled

Successfully assigned openshift-ingress/e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj to ip-10-0-130-54.us-west-1.compute.internal

ai-tenants

e2e-api-a-8a9760

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-a-8a9760: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97 to ip-10-0-138-21.us-west-1.compute.internal

kuadrant-system

authorino-64c8bb8959-2zlbz

Scheduled

Successfully assigned kuadrant-system/authorino-64c8bb8959-2zlbz to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8 to ip-10-0-130-54.us-west-1.compute.internal

kuadrant-system

authorino-6f6944bcdd-7whnd

Scheduled

Successfully assigned kuadrant-system/authorino-6f6944bcdd-7whnd to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t to ip-10-0-138-21.us-west-1.compute.internal

opendatahub-operator-system

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Scheduled

Successfully assigned opendatahub-operator-system/opendatahub-operator-controller-manager-5965b9bff7-hjg5w to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7876bfcb74-sh5qw

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-sh5qw to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz to ip-10-0-138-21.us-west-1.compute.internal

opendatahub

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

Scheduled

Successfully assigned opendatahub/usage-logs-tenancy-proxy-9c5d49bcd-xg8kb to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9 to ip-10-0-130-54.us-west-1.compute.internal

kuadrant-system

authorino-779d94779c-7l5gf

Scheduled

Successfully assigned kuadrant-system/authorino-779d94779c-7l5gf to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Scheduled

Successfully assigned openshift-ingress/e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

test-curl-7695-c60e8b

Scheduled

Successfully assigned opendatahub/test-curl-7695-c60e8b to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx to ip-10-0-140-68.us-west-1.compute.internal

kuadrant-system

authorino-operator-7b7bb4bc77-6jbbb

Scheduled

Successfully assigned kuadrant-system/authorino-operator-7b7bb4bc77-6jbbb to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

test-curl-7695-39603a

Scheduled

Successfully assigned opendatahub/test-curl-7695-39603a to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

test-curl-7695-0ff146

Scheduled

Successfully assigned opendatahub/test-curl-7695-0ff146 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

test-curl-7689-899601

Scheduled

Successfully assigned opendatahub/test-curl-7689-899601 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Scheduled

Successfully assigned openshift-ingress/e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

test-curl-7689-6a6333

Scheduled

Successfully assigned opendatahub/test-curl-7689-6a6333 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg to ip-10-0-138-21.us-west-1.compute.internal

kuadrant-system

dns-operator-controller-manager-8557cbdb84-5hrk6

Scheduled

Successfully assigned kuadrant-system/dns-operator-controller-manager-8557cbdb84-5hrk6 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

test-curl-7686-99155e

Scheduled

Successfully assigned opendatahub/test-curl-7686-99155e to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

test-curl-7686-3f6539

Scheduled

Successfully assigned opendatahub/test-curl-7686-3f6539 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-7b98bb678c-7dngs

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-7dngs to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-5b4cf47bfb-xjj8z

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-5b4cf47bfb-xjj8z to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-56f8865864-tcnsn

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-56f8865864-tcnsn to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

test-curl-7686-3d97be

Scheduled

Successfully assigned opendatahub/test-curl-7686-3d97be to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

Scheduled

Successfully assigned openshift-ingress/e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

Scheduled

Successfully assigned openshift-ingress/e2e-worker-main-976689-openshift-default-5db554f479-mgddl to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49 to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

test-curl-7686-254a3e

Scheduled

Successfully assigned opendatahub/test-curl-7686-254a3e to ip-10-0-138-21.us-west-1.compute.internal

kuadrant-system

kuadrant-console-plugin-66c4fb575d-pwpvc

Scheduled

Successfully assigned kuadrant-system/kuadrant-console-plugin-66c4fb575d-pwpvc to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Scheduled

Successfully assigned openshift-ingress/e2e-api-b-d855db-openshift-default-577cbd8699-6djwm to ip-10-0-138-21.us-west-1.compute.internal

llm

e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

Scheduled

Successfully assigned llm/e2e-trlp-test-simulated-kserve-69b588897d-gh8jf to ip-10-0-138-21.us-west-1.compute.internal

llm

e2e-same-id-b-fff8f9af

ModelNameConflictResolved

Model identity is no longer shared with any other MaaSModelRef in this namespace

llm

e2e-same-id-b-fff8f9af

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-a-fff8f9af

openshift-ingress

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

test-curl-7686-13cf87

Scheduled

Successfully assigned opendatahub/test-curl-7686-13cf87 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq to ip-10-0-138-21.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-b9zd5

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-b9zd5 to ip-10-0-140-68.us-west-1.compute.internal

openshift-jobset-operator

jobset-controller-manager-6ddb96fb5-b4jlm

Scheduled

Successfully assigned openshift-jobset-operator/jobset-controller-manager-6ddb96fb5-b4jlm to ip-10-0-138-21.us-west-1.compute.internal

llm

facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

Scheduled

Successfully assigned llm/facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22 to ip-10-0-130-54.us-west-1.compute.internal

llm

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

Scheduled

Successfully assigned llm/e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf to ip-10-0-130-54.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-jpshj

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-678d87766f-jpshj to ip-10-0-140-68.us-west-1.compute.internal

llm

e2e-same-id-a-fff8f9af

ModelNameConflict

Model identity "publishers/llm/models/test/e2e-same-model-id" is shared with 1 other MaaSModelRef in this namespace: e2e-same-id-b-fff8f9af

opendatahub

test-curl-7686-046fbf

Scheduled

Successfully assigned opendatahub/test-curl-7686-046fbf to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

odh-model-controller-769875db56-4nlhs

Scheduled

Successfully assigned opendatahub/odh-model-controller-769875db56-4nlhs to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

model-serving-api-75bd89fdfc-8p824

Scheduled

Successfully assigned opendatahub/model-serving-api-75bd89fdfc-8p824 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-6r85z

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-6r85z to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

Scheduled

Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5 to ip-10-0-138-21.us-west-1.compute.internal

opendatahub

maas-controller-9fc89bb6-ccp7t

Scheduled

Successfully assigned opendatahub/maas-controller-9fc89bb6-ccp7t to ip-10-0-140-68.us-west-1.compute.internal

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-99hfg

Scheduled

Successfully assigned kuadrant-system/kuadrant-operator-controller-manager-cf8469d7b-99hfg to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-54ddd7df84-r5p2d

Scheduled

Successfully assigned openshift-ingress/payload-processing-54ddd7df84-r5p2d to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-55fbbbf9dc-q2fb6

Scheduled

Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-q2fb6 to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

maas-controller-64d475bcc9-z7whx

Scheduled

Successfully assigned opendatahub/maas-controller-64d475bcc9-z7whx to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-6759564b6c-wpg92

Scheduled

Successfully assigned openshift-ingress/payload-processing-6759564b6c-wpg92 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

maas-controller-64d475bcc9-m8vd5

Scheduled

Successfully assigned opendatahub/maas-controller-64d475bcc9-m8vd5 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-api-a-8a9760-998649d7c-lz57c

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-a-8a9760-998649d7c-lz57c to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-api-b-d855db-76f476689-zjv2j

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-api-b-d855db-76f476689-zjv2j to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

maas-controller-64d475bcc9-cs8xr

Scheduled

Successfully assigned opendatahub/maas-controller-64d475bcc9-cs8xr to ip-10-0-138-21.us-west-1.compute.internal

kuadrant-system

limitador-limitador-74dfcfb894-62f4s

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-74dfcfb894-62f4s to ip-10-0-138-21.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-869d5dcc5d-82cfd

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-869d5dcc5d-82cfd to ip-10-0-138-21.us-west-1.compute.internal

opendatahub

llmisvc-controller-manager-6466c554d8-446vb

Scheduled

Successfully assigned opendatahub/llmisvc-controller-manager-6466c554d8-446vb to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

kserve-module-controller-manager-7566cf5fc8-pwf6c

Scheduled

Successfully assigned opendatahub/kserve-module-controller-manager-7566cf5fc8-pwf6c to ip-10-0-138-21.us-west-1.compute.internal

opendatahub

kserve-controller-manager-776cc66d47-k9s7l

Scheduled

Successfully assigned opendatahub/kserve-controller-manager-776cc66d47-k9s7l to ip-10-0-140-68.us-west-1.compute.internal

opendatahub

ai-gateway-controller-7d85675f98-87wcp

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-7d85675f98-87wcp to ip-10-0-130-54.us-west-1.compute.internal

opendatahub

ai-gateway-controller-575f85f67b-6vdks

Scheduled

Successfully assigned opendatahub/ai-gateway-controller-575f85f67b-6vdks to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-derive-030936cd-5878568bf9-zblpr

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-derive-030936cd-5878568bf9-zblpr to ip-10-0-130-54.us-west-1.compute.internal

kuadrant-system

limitador-limitador-7c9b87d574-b57dc

Scheduled

Successfully assigned kuadrant-system/limitador-limitador-7c9b87d574-b57dc to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Scheduled

Successfully assigned openshift-ingress/e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Scheduled

Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-j7glh to ip-10-0-138-21.us-west-1.compute.internal

llm

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Scheduled

Successfully assigned llm/premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-581487-f88795567-8kgw2 to ip-10-0-130-54.us-west-1.compute.internal

kuadrant-system

limitador-operator-controller-manager-5778bb4449-q7mrd

Scheduled

Successfully assigned kuadrant-system/limitador-operator-controller-manager-5778bb4449-q7mrd to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

Scheduled

Successfully assigned openshift-ingress/e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-zq7gr to ip-10-0-138-21.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-985777bbf-wxk2d

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-985777bbf-wxk2d to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-xqsk9 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-x4lgh to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

Scheduled

Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq to ip-10-0-130-54.us-west-1.compute.internal

openshift-jobset-operator

jobset-operator-674cd59dc5-thpdh

Scheduled

Successfully assigned openshift-jobset-operator/jobset-operator-674cd59dc5-thpdh to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-68cbf89484-6x6nh

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-68cbf89484-6x6nh to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Scheduled

Successfully assigned openshift-ingress/e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-wvfkd to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-vktw8

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-vktw8 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466 to ip-10-0-140-68.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-985777bbf-ln7cv

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-985777bbf-ln7cv to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-sjcrx to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-r9phn to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-r5wn2 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-qpmzx to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-qb2mm to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pn2tc to ip-10-0-138-21.us-west-1.compute.internal

llm

e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

Scheduled

Successfully assigned llm/e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-p4lr6 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-njl8w to ip-10-0-140-68.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-866d5b9c86-vpmlw

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-866d5b9c86-vpmlw to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-n2vdj to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-kvv59

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-kvv59 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-krv87

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-krv87 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-j4f5x to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw to ip-10-0-130-54.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-866d5b9c86-qm2hn

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-866d5b9c86-qm2hn to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-hjj86 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-g6cln

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt to ip-10-0-140-68.us-west-1.compute.internal

llm

e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

Scheduled

Successfully assigned llm/e2e-distinct-simulated-kserve-7b998b79d9-gzgw9 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-g2829

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g2829 to ip-10-0-138-21.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-866d5b9c86-lp22f

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-866d5b9c86-lp22f to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-fxl5k to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-dwr2h to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-d9bdq to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-c69zs

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-c69zs to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-9tp28

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-9tp28 to ip-10-0-130-54.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-866d5b9c86-dxh9q

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-866d5b9c86-dxh9q to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Scheduled

Successfully assigned openshift-ingress/e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-9hv44

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-9hv44 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5 to ip-10-0-130-54.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-7b968cbf54-hrgz2

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7b968cbf54-hrgz2 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j to ip-10-0-140-68.us-west-1.compute.internal

llm

e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

Scheduled

Successfully assigned llm/e2e-embedding-simulated-kserve-67bf594cd6-pmrj9 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

Scheduled

Successfully assigned openshift-ingress/e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

kube-auth-proxy-68cbf89484-cmqtk

Scheduled

Successfully assigned openshift-ingress/kube-auth-proxy-68cbf89484-cmqtk to ip-10-0-138-21.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-595cb556d4-tfnm9

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-595cb556d4-tfnm9 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Scheduled

Successfully assigned openshift-ingress/e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Scheduled

Successfully assigned openshift-ingress/e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-8ckbd to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt to ip-10-0-130-54.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-787d5c596d-gsfjj

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-787d5c596d-gsfjj to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

Scheduled

Successfully assigned openshift-ingress/e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5 to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Scheduled

Successfully assigned openshift-ingress/e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr to ip-10-0-130-54.us-west-1.compute.internal

llm

e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

Scheduled

Successfully assigned llm/e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

router-default-78b77c8f77-457bj

Scheduled

Successfully assigned openshift-ingress/router-default-78b77c8f77-457bj to ip-10-0-130-54.us-west-1.compute.internal

llm

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

Scheduled

Successfully assigned llm/e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6 to ip-10-0-130-54.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-7446cdb47f-xqntm

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-7446cdb47f-xqntm to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76 to ip-10-0-130-54.us-west-1.compute.internal

llm

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Scheduled

Successfully assigned llm/e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-8c8g9 to ip-10-0-140-68.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-6bd55dd7d7-7swd4

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-6bd55dd7d7-7swd4 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Scheduled

Successfully assigned openshift-ingress/e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9 to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-7x5qn to ip-10-0-130-54.us-west-1.compute.internal

ai-tenants

e2e-api-b-d855db

APIKeyCleanupSkipped

Skipping API-key cleanup for AITenant ai-tenants/e2e-api-b-d855db: MaasTenantConfig is not available to restore revocation operands

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-7jm7x to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Scheduled

Successfully assigned openshift-ingress/e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-6zskg

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-6zskg to ip-10-0-138-21.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd to ip-10-0-130-54.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-66c9f9b6bc-q27lf

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-66c9f9b6bc-q27lf to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-476fj

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-476fj to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-2tj59

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-2tj59 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

maas-default-gateway-openshift-default-7d9897b54c-2np4m

Scheduled

Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-2np4m to ip-10-0-140-68.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-64954bd78b-zr94k

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-64954bd78b-zr94k to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2 to ip-10-0-140-68.us-west-1.compute.internal

openshift-ingress

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Scheduled

Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb to ip-10-0-138-21.us-west-1.compute.internal

models-as-a-service

payload-processing-external-model-5bcd7997-lbz5s

Scheduled

Successfully assigned models-as-a-service/payload-processing-external-model-5bcd7997-lbz5s to ip-10-0-130-54.us-west-1.compute.internal

openshift-ingress

router-default-78b77c8f77-457bj

FailedScheduling

no nodes available to schedule pods

openshift-ingress

router-default-78b77c8f77-457bj

FailedScheduling

no nodes available to schedule pods

openshift-ingress

service-controller

router-default

UnAvailableLoadBalancer

There are no available nodes for LoadBalancer

openshift-ingress

service-controller

router-default

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

router-default

ScalingReplicaSet

Scaled up replica set router-default-78b77c8f77 from 0 to 1

openshift-ingress

service-controller

router-default

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

replicaset-controller

router-default-78b77c8f77

SuccessfulCreate

Created pod: router-default-78b77c8f77-457bj
(x3)

openshift-ingress

service-controller

router-default

UpdatedLoadBalancer

Updated load balancer with new hosts
(x5)

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found
(x5)

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x4)

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

FailedMount

MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt
(x5)

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

FailedMount

MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

Started

Container started

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

Created

Container created

openshift-ingress

kubelet

router-default-78b77c8f77-457bj

Pulled

Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod

openshift-ingress

multus

router-default-78b77c8f77-457bj

AddedInterface

Add eth0 [10.132.0.11/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

maas-default-gateway

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1

openshift-ingress

multus

istiod-openshift-gateway-7876bfcb74-sh5qw

AddedInterface

Add eth0 [10.132.0.25/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulCreate

Created pod: istiod-openshift-gateway-7876bfcb74-sh5qw

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.178s (4.178s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-8ckbd

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuringLoadBalancer

Ensuring load balancer

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 0 to 1

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

AddedInterface

Add eth0 [10.134.0.23/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

replicaset-controller

openshift-ai-inference-openshift-ai-inference-6dd6bfb767

SuccessfulCreate

Created pod: openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

openshift-ingress

deployment-controller

openshift-ai-inference-openshift-ai-inference

ScalingReplicaSet

Scaled up replica set openshift-ai-inference-openshift-ai-inference-6dd6bfb767 from 0 to 1

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

AddedInterface

Add eth0 [10.134.0.24/23] from ovn-kubernetes

openshift-ingress

service-controller

openshift-ai-inference-openshift-ai-inference

EnsuredLoadBalancer

Ensured load balancer

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.517s (13.517s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

Created

Container created

openshift-ingress

kubelet

openshift-ai-inference-openshift-ai-inference-6dd6bfb767-bkcz7

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.227s (13.227s including waiting). Image size: 183555333 bytes.
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

openshift-ai-inference-openshift-ai-inference

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

RequirementsNotMet

one or more requirements couldn't be found

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-5965b9bff7

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-5965b9bff7-zmv2g

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-5965b9bff7

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-5965b9bff7-hjg5w

opendatahub-operator-system

deployment-controller

opendatahub-operator-controller-manager

ScalingReplicaSet

Scaled up replica set opendatahub-operator-controller-manager-5965b9bff7 from 0 to 3

opendatahub-operator-system

replicaset-controller

opendatahub-operator-controller-manager-5965b9bff7

SuccessfulCreate

Created pod: opendatahub-operator-controller-manager-5965b9bff7-vr9xt

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

AllRequirementsMet

all requirements found, attempting install

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

AddedInterface

Add eth0 [10.132.0.26/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

Webhook install failed: conversionWebhook not ready

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

AddedInterface

Add eth0 [10.133.0.23/23] from ovn-kubernetes

opendatahub-operator-system

multus

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

AddedInterface

Add eth0 [10.134.0.29/23] from ovn-kubernetes

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

RequirementsUnknown

requirements not yet checked
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

waiting for install components to report healthy

opendatahub-operator-system

operator-lifecycle-manager

install-g27wh

AppliedWithWarnings

1 warning(s) generated during installation of operator "opendatahub-operator.v3.6.0-ea.2" (CustomResourceDefinition "hardwareprofiles.infrastructure.opendatahub.io"): infrastructure.opendatahub.io/v1alpha1 is deprecated; please use infrastructure.opendatahub.io/v1
(x2)

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallWaiting

installing: waiting for deployment opendatahub-operator-controller-manager to become ready: deployment "opendatahub-operator-controller-manager" not available: Deployment does not have minimum availability.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.298s (3.298s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 3.378s (3.378s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 4.124s (4.124s including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 840ms (840ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Created

Container created

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-zmv2g

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Pulling

Pulling image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2"

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 809ms (809ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-vr9xt

Started

Container started

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Pulled

Successfully pulled image "quay.io/opendatahub/opendatahub-operator:v3.6.0-ea.2" in 759ms (759ms including waiting). Image size: 273709574 bytes.

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Created

Container created

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsUnknown

requirements not yet checked

opendatahub-operator-system

kubelet

opendatahub-operator-controller-manager-5965b9bff7-hjg5w

Started

Container started

opendatahub-operator-system

opendatahub-operator-controller-manager-5965b9bff7-vr9xt_96f1ae9b-39ac-4315-b25b-241b8cfe3f01

07ed84f7.opendatahub.io

LeaderElection

opendatahub-operator-controller-manager-5965b9bff7-vr9xt_96f1ae9b-39ac-4315-b25b-241b8cfe3f01 became leader

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

deployment-controller

limitador-operator-controller-manager

ScalingReplicaSet

Scaled up replica set limitador-operator-controller-manager-5778bb4449 from 0 to 1

kuadrant-system

replicaset-controller

limitador-operator-controller-manager-5778bb4449

SuccessfulCreate

Created pod: limitador-operator-controller-manager-5778bb4449-q7mrd

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallWaiting

installing: waiting for deployment limitador-operator-controller-manager to become ready: deployment "limitador-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

RequirementsNotMet

one or more requirements couldn't be found

kuadrant-system

multus

limitador-operator-controller-manager-5778bb4449-q7mrd

AddedInterface

Add eth0 [10.134.0.30/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-q7mrd

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e"

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

AllRequirementsMet

all requirements found, attempting install

openshift-jobset-operator

replicaset-controller

jobset-operator-674cd59dc5

SuccessfulCreate

Created pod: jobset-operator-674cd59dc5-thpdh

openshift-jobset-operator

deployment-controller

jobset-operator

ScalingReplicaSet

Scaled up replica set jobset-operator-674cd59dc5 from 0 to 1

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

waiting for install components to report healthy

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-thpdh

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e"

openshift-jobset-operator

multus

jobset-operator-674cd59dc5-thpdh

AddedInterface

Add eth0 [10.133.0.24/23] from ovn-kubernetes

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallWaiting

installing: waiting for deployment jobset-operator to become ready: deployment "jobset-operator" not available: Deployment does not have minimum availability.

kuadrant-system

deployment-controller

dns-operator-controller-manager

ScalingReplicaSet

Scaled up replica set dns-operator-controller-manager-8557cbdb84 from 0 to 1

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-q7mrd

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9-operator@sha256:7fdc92b22c7d74e68e71ebf54d1c680fc8dbd47198286dc4f7f87b3d4b31472e" in 3.496s (3.496s including waiting). Image size: 177967631 bytes.

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

replicaset-controller

dns-operator-controller-manager-8557cbdb84

SuccessfulCreate

Created pod: dns-operator-controller-manager-8557cbdb84-5hrk6

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-q7mrd

Started

Container started

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-5hrk6

Pulling

Pulling image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a"

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallWaiting

installing: waiting for deployment dns-operator-controller-manager to become ready: deployment "dns-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

dns-operator-controller-manager-8557cbdb84-5hrk6

AddedInterface

Add eth0 [10.134.0.31/23] from ovn-kubernetes

kuadrant-system

limitador-operator-controller-manager-5778bb4449-q7mrd_ee6f7061-0d5c-4d0b-ae28-2e52287e2535

3745a16e.kuadrant.io

LeaderElection

limitador-operator-controller-manager-5778bb4449-q7mrd_ee6f7061-0d5c-4d0b-ae28-2e52287e2535 became leader

kuadrant-system

kubelet

limitador-operator-controller-manager-5778bb4449-q7mrd

Created

Container created

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-thpdh

Started

Container started

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-thpdh

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9-operator@sha256:a258511e57b511c5d2a54fae2b8165669d677fb3a69796aa1c78d224a499033e" in 3.655s (3.655s including waiting). Image size: 224874686 bytes.

openshift-jobset-operator

kubelet

jobset-operator-674cd59dc5-thpdh

Created

Container created

openshift-jobset-operator

openshift-jobset-operator

openshift-jobset-operator-lock

LeaderElection

jobset-operator-674cd59dc5-thpdh_9800d89c-35ed-455c-a54b-b1552219b3c2 became leader

openshift-jobset-operator

operator-lifecycle-manager

jobset-operator.v1.0.1

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-5hrk6

Started

Container started

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-5hrk6

Created

Container created

kuadrant-system

kubelet

dns-operator-controller-manager-8557cbdb84-5hrk6

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/dns-rhel9-operator@sha256:d82688172d312a44d9978eacf7bd9e0402e9ba2967daf70b2ef30707f41faf4a" in 2.52s (2.52s including waiting). Image size: 190804149 bytes.

kuadrant-system

dns-operator-controller-manager-8557cbdb84-5hrk6_e03c153e-a884-4e0c-a09b-b87801f75298

a3f98d6c.kuadrant.io

LeaderElection

dns-operator-controller-manager-8557cbdb84-5hrk6_e03c153e-a884-4e0c-a09b-b87801f75298 became leader

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-cf8469d7b-99hfg

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-cf8469d7b from 0 to 1

opendatahub-operator-system

operator-lifecycle-manager

opendatahub-operator.v3.6.0-ea.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

multus

kuadrant-operator-controller-manager-cf8469d7b-99hfg

AddedInterface

Add eth0 [10.134.0.33/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-99hfg

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e"

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsUnknown

requirements not yet checked

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

RequirementsNotMet

one or more requirements couldn't be found

cert-manager

deployment-controller

cert-manager

ScalingReplicaSet

Scaled up replica set cert-manager-597b95cb84 from 0 to 1

cert-manager

deployment-controller

cert-manager-cainjector

ScalingReplicaSet

Scaled up replica set cert-manager-cainjector-5c57bbd4fc from 0 to 1

kuadrant-system

operator-lifecycle-manager

limitador-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

authorino-operator

ScalingReplicaSet

Scaled up replica set authorino-operator-7b7bb4bc77 from 0 to 1

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallWaiting

installing: waiting for deployment authorino-operator to become ready: deployment "authorino-operator" not available: Deployment does not have minimum availability.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install

kuadrant-system

multus

authorino-operator-7b7bb4bc77-6jbbb

AddedInterface

Add eth0 [10.134.0.34/23] from ovn-kubernetes

kuadrant-system

replicaset-controller

authorino-operator-7b7bb4bc77

SuccessfulCreate

Created pod: authorino-operator-7b7bb4bc77-6jbbb

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

SuccessfulCreate

Created pod: cert-manager-cainjector-5c57bbd4fc-4g7gv
(x8)

cert-manager

replicaset-controller

cert-manager-cainjector-5c57bbd4fc

FailedCreate

Error creating: pods "cert-manager-cainjector-5c57bbd4fc-" is forbidden: error looking up service account cert-manager/cert-manager-cainjector: serviceaccount "cert-manager-cainjector" not found

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-4g7gv

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

kuadrant-system

kuadrant-operator-controller-manager-cf8469d7b-99hfg_0c302224-322d-4c67-bbdf-dac0f3e4a2b0

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-cf8469d7b-99hfg_0c302224-322d-4c67-bbdf-dac0f3e4a2b0 became leader

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-6jbbb

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589"

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-99hfg

Created

Container created

cert-manager

multus

cert-manager-cainjector-5c57bbd4fc-4g7gv

AddedInterface

Add eth0 [10.133.0.25/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-99hfg

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-99hfg

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" in 4.462s (4.462s including waiting). Image size: 424244516 bytes.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-6jbbb

Created

Container created

kuadrant-system

authorino-operator-7b7bb4bc77-6jbbb_01e9ca5b-27a9-4033-98c5-ad533bebcd02

aac3a15d.authorino.kuadrant.io

LeaderElection

authorino-operator-7b7bb4bc77-6jbbb_01e9ca5b-27a9-4033-98c5-ad533bebcd02 became leader

kuadrant-system

operator-lifecycle-manager

dns-operator.v1.4.2

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-6jbbb

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9-operator@sha256:8a8cc5fb104032c23096f3c9cf2fbc3636d343ee7ee7974e51fe1f10d3a57589" in 2.175s (2.175s including waiting). Image size: 167738465 bytes.

kuadrant-system

kubelet

authorino-operator-7b7bb4bc77-6jbbb

Started

Container started

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-metrics-reader because it was missing

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-4g7gv

Started

Container started

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceCreated

Created Service/jobset-webhook-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-manager-role because it was missing

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-4g7gv

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 4.234s (4.234s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-cainjector-5c57bbd4fc-4g7gv

Created

Container created

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceAccountCreated

Created ServiceAccount/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-manager-rolebinding because it was missing

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-rxfxj

Created

Container created

cert-manager

multus

cert-manager-webhook-5dbbbf75cc-rxfxj

AddedInterface

Add eth0 [10.133.0.26/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-rxfxj

Pulled

Container image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" already present on machine and can be accessed by the pod

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleCreated

Created ClusterRole.rbac.authorization.k8s.io/jobset-proxy-role because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-leader-election-role -n openshift-jobset-operator because it was missing

cert-manager

deployment-controller

cert-manager-webhook

ScalingReplicaSet

Scaled up replica set cert-manager-webhook-5dbbbf75cc from 0 to 1

cert-manager

replicaset-controller

cert-manager-webhook-5dbbbf75cc

SuccessfulCreate

Created pod: cert-manager-webhook-5dbbbf75cc-rxfxj

cert-manager

kubelet

cert-manager-webhook-5dbbbf75cc-rxfxj

Started

Container started

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-metrics-reader-rolebinding because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ClusterRoleBindingCreated

Created ClusterRoleBinding.rbac.authorization.k8s.io/jobset-proxy-rolebinding because it was missing

openshift-ingress

deployment-controller

data-science-gateway-data-science-gateway-class

ScalingReplicaSet

Scaled up replica set data-science-gateway-data-science-gateway-class-858f55f4d9 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-manager-secrets-role -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

ServiceCreated

Created Service/jobset-controller-manager-metrics-service -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-manager-secrets-rolebinding -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator-jobsetoperatorstaticresources-jobsetoperatorstaticresources-staticresources

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-leader-election-rolebinding -n openshift-jobset-operator because it was missing

openshift-ingress

deployment-controller

kube-auth-proxy

ScalingReplicaSet

Scaled up replica set kube-auth-proxy-68cbf89484 from 0 to 2

openshift-ingress

replicaset-controller

kube-auth-proxy-68cbf89484

SuccessfulCreate

Created pod: kube-auth-proxy-68cbf89484-6x6nh

openshift-ingress

replicaset-controller

kube-auth-proxy-68cbf89484

SuccessfulCreate

Created pod: kube-auth-proxy-68cbf89484-cmqtk

openshift-ingress

replicaset-controller

data-science-gateway-data-science-gateway-class-858f55f4d9

SuccessfulCreate

Created pod: data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd

openshift-ingress

multus

kube-auth-proxy-68cbf89484-6x6nh

AddedInterface

Add eth0 [10.132.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-6x6nh

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd

AddedInterface

Add eth0 [10.134.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-cmqtk

Pulling

Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035"

openshift-ingress

multus

kube-auth-proxy-68cbf89484-cmqtk

AddedInterface

Add eth0 [10.133.0.27/23] from ovn-kubernetes

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd

Started

Container started

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd

Created

Container created

openshift-ingress

kubelet

data-science-gateway-data-science-gateway-class-858f55f4d9cr5hd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

kuadrant-system

deployment-controller

kuadrant-console-plugin

ScalingReplicaSet

Scaled up replica set kuadrant-console-plugin-66c4fb575d from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-console-plugin-66c4fb575d

SuccessfulCreate

Created pod: kuadrant-console-plugin-66c4fb575d-pwpvc

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-74dfcfb894 from 0 to 1
(x12)

cert-manager

replicaset-controller

cert-manager-597b95cb84

FailedCreate

Error creating: pods "cert-manager-597b95cb84-" is forbidden: error looking up service account cert-manager/cert-manager: serviceaccount "cert-manager" not found

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulCreate

Created pod: limitador-limitador-74dfcfb894-62f4s

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pwpvc

Pulling

Pulling image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044"

kuadrant-system

multus

kuadrant-console-plugin-66c4fb575d-pwpvc

AddedInterface

Add eth0 [10.134.0.36/23] from ovn-kubernetes
(x4)

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreateFailed

Failed to create Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator: Internal error occurred: failed calling webhook "webhook.cert-manager.io": failed to call webhook: Post "https://cert-manager-webhook.cert-manager.svc:443/validate?timeout=30s": no endpoints available for service "cert-manager-webhook"

kuadrant-system

multus

limitador-limitador-74dfcfb894-62f4s

AddedInterface

Add eth0 [10.133.0.28/23] from ovn-kubernetes

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-62f4s

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-6x6nh

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.796s (2.796s including waiting). Image size: 188904677 bytes.

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-cmqtk

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.938s (2.938s including waiting). Image size: 188904677 bytes.

kuadrant-system

kubelet

authorino-57f7f8659f-xjhwv

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-57f7f8659f-xjhwv

AddedInterface

Add eth0 [10.134.0.37/23] from ovn-kubernetes

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled up replica set limitador-limitador-7c9b87d574 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-56c6c9ddd5 from 0 to 1

kuadrant-system

replicaset-controller

limitador-limitador-7c9b87d574

SuccessfulCreate

Created pod: limitador-limitador-7c9b87d574-b57dc

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-cmqtk

Created

Container created

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-57f7f8659f from 0 to 1

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-6x6nh

Started

Container started

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulCreate

Created pod: authorino-56c6c9ddd5-ggd22

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-cmqtk

Started

Container started

kuadrant-system

multus

authorino-56c6c9ddd5-ggd22

AddedInterface

Add eth0 [10.134.0.38/23] from ovn-kubernetes

kuadrant-system

multus

limitador-limitador-7c9b87d574-b57dc

AddedInterface

Add eth0 [10.133.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

kube-auth-proxy-68cbf89484-6x6nh

Created

Container created

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulCreate

Created pod: authorino-57f7f8659f-xjhwv

kuadrant-system

kubelet

authorino-56c6c9ddd5-ggd22

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-b57dc

Pulling

Pulling image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751"

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: deployment changed old hash=6q1WY8Uae4JZmLF4M4Zj2lZF5tKn8wEFi4k5Ve, new hash=88W7Jv6nAXJm3xNFHpPpBPB6INrNa0R4IQ7iYv

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" waiting for 1 outdated replica(s) to be terminated

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-jpshj

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-b57dc

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-jpshj

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-jpshj

AddedInterface

Add eth0 [10.134.0.39/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-jpshj

Created

Container created

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-62f4s

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 3.404s (3.404s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-62f4s

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-b57dc

Created

Container created

kuadrant-system

kubelet

limitador-limitador-7c9b87d574-b57dc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/limitador-rhel9@sha256:793a38ce1b5690b99bc20fb7fca3304727d5fb21fcf440ab88f3e7af5778a751" in 2.09s (2.09s including waiting). Image size: 133379307 bytes.

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-62f4s

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-jpshj

Started

Container started

kuadrant-system

replicaset-controller

authorino-57f7f8659f

SuccessfulDelete

Deleted pod: authorino-57f7f8659f-xjhwv

kuadrant-system

kubelet

authorino-57f7f8659f-xjhwv

Started

Container started

kuadrant-system

kubelet

authorino-57f7f8659f-xjhwv

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-xjhwv

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 3.1s (3.1s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-56c6c9ddd5-ggd22

Started

Container started

kuadrant-system

kubelet

authorino-56c6c9ddd5-ggd22

Created

Container created

kuadrant-system

kubelet

authorino-56c6c9ddd5-ggd22

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 2.929s (2.929s including waiting). Image size: 204096810 bytes.

kuadrant-system

operator-lifecycle-manager

authorino-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-57f7f8659f from 1 to 0

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pwpvc

Started

Container started

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pwpvc

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/rhcl-console-plugin-rhel9@sha256:c8550de6790d58c5fa993bf83d41ef693d99a5739fcc0eed3fcaa8d5a7658044" in 6.218s (6.218s including waiting). Image size: 342538866 bytes.

kuadrant-system

kubelet

kuadrant-console-plugin-66c4fb575d-pwpvc

Created

Container created

kuadrant-system

kubelet

authorino-57f7f8659f-xjhwv

Killing

Stopping container authorino

cert-manager

replicaset-controller

cert-manager-597b95cb84

SuccessfulCreate

Created pod: cert-manager-597b95cb84-kbvqq

cert-manager

multus

cert-manager-597b95cb84-kbvqq

AddedInterface

Add eth0 [10.132.0.28/23] from ovn-kubernetes

cert-manager

kubelet

cert-manager-597b95cb84-kbvqq

Pulling

Pulling image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d"

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

cert-manager

kubelet

cert-manager-597b95cb84-kbvqq

Started

Container started

cert-manager

kubelet

cert-manager-597b95cb84-kbvqq

Pulled

Successfully pulled image "registry.redhat.io/cert-manager/jetstack-cert-manager-rhel9@sha256:b3305a06bff4a344ba1b19c8142d7e605fc723841981c97aab2310ff8a82c66d" in 3.736s (3.736s including waiting). Image size: 309650901 bytes.

cert-manager

kubelet

cert-manager-597b95cb84-kbvqq

Created

Container created

kuadrant-system

deployment-controller

limitador-limitador

ScalingReplicaSet

Scaled down replica set limitador-limitador-74dfcfb894 from 1 to 0

kuadrant-system

kubelet

kuadrant-operator-controller-manager-cf8469d7b-99hfg

Killing

Stopping container manager

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-cf8469d7b from 1 to 0

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-cf8469d7b

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-cf8469d7b-99hfg

kuadrant-system

replicaset-controller

limitador-limitador-74dfcfb894

SuccessfulDelete

Deleted pod: limitador-limitador-74dfcfb894-62f4s

kuadrant-system

kubelet

limitador-limitador-74dfcfb894-62f4s

Killing

Stopping container limitador

opendatahub

multus

kserve-module-controller-manager-7566cf5fc8-pwf6c

AddedInterface

Add eth0 [10.133.0.30/23] from ovn-kubernetes

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pwf6c

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189"

opendatahub

replicaset-controller

kserve-module-controller-manager-7566cf5fc8

SuccessfulCreate

Created pod: kserve-module-controller-manager-7566cf5fc8-pwf6c

opendatahub

deployment-controller

kserve-module-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-module-controller-manager-7566cf5fc8 from 0 to 1

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pwf6c

Created

Container created

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pwf6c

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-module-operator@sha256:b8becaa017a2a2ec9af002989e8f14a8a2f96ffad292f37a61569343b1101189" in 1.859s (1.859s including waiting). Image size: 195729000 bytes.

opendatahub

kubelet

kserve-module-controller-manager-7566cf5fc8-pwf6c

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

kube-auth-proxy

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

data-science-gateway-data-science-gateway-class

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-jpshj_9efe5d7e-98a4-4037-98b7-a7fee61fb26b

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-jpshj_9efe5d7e-98a4-4037-98b7-a7fee61fb26b became leader

opendatahub

kserve-module-controller-manager-7566cf5fc8-pwf6c_832051bf-3056-4b85-b1b5-680e66b87cba

kserve-module-controller-manager

LeaderElection

kserve-module-controller-manager-7566cf5fc8-pwf6c_832051bf-3056-4b85-b1b5-680e66b87cba became leader

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-9fc89bb6 from 0 to 1

opendatahub

replicaset-controller

maas-controller-9fc89bb6

SuccessfulCreate

Created pod: maas-controller-9fc89bb6-ccp7t

opendatahub

multus

maas-controller-9fc89bb6-ccp7t

AddedInterface

Add eth0 [10.134.0.41/23] from ovn-kubernetes

opendatahub

multus

maas-controller-64d475bcc9-z7whx

AddedInterface

Add eth0 [10.133.0.31/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-9fc89bb6-ccp7t

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159"

opendatahub

replicaset-controller

maas-controller-64d475bcc9

SuccessfulCreate

Created pod: maas-controller-64d475bcc9-z7whx

opendatahub

kubelet

maas-controller-64d475bcc9-z7whx

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159" in 6.202s (6.202s including waiting). Image size: 200094742 bytes.

opendatahub

kubelet

maas-controller-9fc89bb6-ccp7t

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159" in 6.768s (6.768s including waiting). Image size: 200094742 bytes.

opendatahub

kubelet

maas-controller-9fc89bb6-ccp7t

Created

Container created

opendatahub

kubelet

maas-controller-9fc89bb6-ccp7t

Started

Container started

opendatahub

maas-controller-9fc89bb6-ccp7t_923954a5-a5de-47c1-9459-962205897fd3

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-9fc89bb6-ccp7t_923954a5-a5de-47c1-9459-962205897fd3 became leader

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

FailedMount

MountVolume.SetUp failed for volume "proxy-source" : configmap "usage-tenancy-proxy-source" not found

opendatahub

deployment-controller

usage-logs-tenancy-proxy

ScalingReplicaSet

Scaled up replica set usage-logs-tenancy-proxy-9c5d49bcd from 0 to 1

opendatahub

replicaset-controller

usage-logs-tenancy-proxy-9c5d49bcd

SuccessfulCreate

Created pod: usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

Pulling

Pulling image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172"

opendatahub

multus

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

AddedInterface

Add eth0 [10.133.0.32/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6r85z

Created

Container created

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-9fc89bb6 from 1 to 0

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-6r85z

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1

opendatahub

replicaset-controller

maas-controller-9fc89bb6

SuccessfulDelete

Deleted pod: maas-controller-9fc89bb6-ccp7t

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6r85z

Started

Container started

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-6r85z

AddedInterface

Add eth0 [10.132.0.29/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6r85z

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod

opendatahub

kubelet

maas-controller-9fc89bb6-ccp7t

Killing

Stopping container manager
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6r85z

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-6r85z

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7876bfcb74-sh5qw

Killing

Stopping container discovery

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7876bfcb74

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7876bfcb74-sh5qw

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0

opendatahub

kubelet

kserve-controller-manager-776cc66d47-k9s7l

Pulling

Pulling image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d"

opendatahub

replicaset-controller

model-serving-api-75bd89fdfc

SuccessfulCreate

Created pod: model-serving-api-75bd89fdfc-8p824

opendatahub

deployment-controller

model-serving-api

ScalingReplicaSet

Scaled up replica set model-serving-api-75bd89fdfc from 0 to 1

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-869d5dcc5d from 0 to 1

opendatahub

deployment-controller

kserve-controller-manager

ScalingReplicaSet

Scaled up replica set kserve-controller-manager-776cc66d47 from 0 to 1

opendatahub

replicaset-controller

kserve-controller-manager-776cc66d47

SuccessfulCreate

Created pod: kserve-controller-manager-776cc66d47-k9s7l

opendatahub

multus

kserve-controller-manager-776cc66d47-k9s7l

AddedInterface

Add eth0 [10.134.0.42/23] from ovn-kubernetes

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulCreate

Created pod: llmisvc-controller-manager-869d5dcc5d-82cfd

opendatahub

deployment-controller

odh-model-controller

ScalingReplicaSet

Scaled up replica set odh-model-controller-769875db56 from 0 to 1

opendatahub

replicaset-controller

odh-model-controller-769875db56

SuccessfulCreate

Created pod: odh-model-controller-769875db56-4nlhs

opendatahub

multus

llmisvc-controller-manager-869d5dcc5d-82cfd

AddedInterface

Add eth0 [10.133.0.33/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-82cfd

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

Started

Container started

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

Pulled

Successfully pulled image "registry.redhat.io/ubi9/python-312@sha256:f6713d327d37e654a443752e6654b5aab88f31690e1161eed9c34dd837870172" in 13.467s (13.467s including waiting). Image size: 1089502687 bytes.

opendatahub

kubelet

model-serving-api-75bd89fdfc-8p824

Pulling

Pulling image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8"

opendatahub

multus

model-serving-api-75bd89fdfc-8p824

AddedInterface

Add eth0 [10.132.0.30/23] from ovn-kubernetes

opendatahub

multus

odh-model-controller-769875db56-4nlhs

AddedInterface

Add eth0 [10.134.0.43/23] from ovn-kubernetes

opendatahub

kubelet

usage-logs-tenancy-proxy-9c5d49bcd-xg8kb

Created

Container created

opendatahub

kubelet

odh-model-controller-769875db56-4nlhs

Pulling

Pulling image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775"

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-82cfd

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-8p824

Created

Container created

opendatahub

kubelet

model-serving-api-75bd89fdfc-8p824

Started

Container started

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-82cfd

Started

Container started

opendatahub

kubelet

model-serving-api-75bd89fdfc-8p824

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-serving-api@sha256:dc6ced7ac752d3770e31e3e83f4f9a6c793de5b9b04dde0d194d021020cef8e8" in 2.068s (2.068s including waiting). Image size: 184925761 bytes.

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-82cfd

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.483s (2.483s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-4nlhs

Pulled

Successfully pulled image "quay.io/opendatahub/odh-model-controller@sha256:94f7f4383ef91d10d60d82c0858a60158a02915fdd3000d507210139faaac775" in 3.277s (3.277s including waiting). Image size: 236755567 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-4nlhs

Started

Container started

opendatahub

kubelet

kserve-controller-manager-776cc66d47-k9s7l

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-k9s7l

Pulled

Successfully pulled image "quay.io/opendatahub/kserve-controller@sha256:6e72a4ac896e998acf222204e10868106932adfbfd0dafdc0a9c1e4d58d0627d" in 3.322s (3.322s including waiting). Image size: 247676581 bytes.

opendatahub

kubelet

odh-model-controller-769875db56-4nlhs

Created

Container created

opendatahub

kubelet

kserve-controller-manager-776cc66d47-k9s7l

Started

Container started

opendatahub

odh-model-controller-769875db56-4nlhs_4f2f0f33-8b04-4c7a-8232-c1d206969db4

odh-model-controller.opendatahub.io

LeaderElection

odh-model-controller-769875db56-4nlhs_4f2f0f33-8b04-4c7a-8232-c1d206969db4 became leader

opendatahub

llmisvc-controller-manager-869d5dcc5d-82cfd_367bfbb8-7718-44fe-92c8-82cbfbd501dd

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-869d5dcc5d-82cfd_367bfbb8-7718-44fe-92c8-82cbfbd501dd became leader

opendatahub

kserve-controller-manager-776cc66d47-k9s7l_b5e3d2c9-f0e6-4c4c-850f-41b6aa0a549f

kserve-controller-manager-leader-lock

LeaderElection

kserve-controller-manager-776cc66d47-k9s7l_b5e3d2c9-f0e6-4c4c-850f-41b6aa0a549f became leader

opendatahub

maas-controller-64d475bcc9-z7whx_73595d81-d909-4423-bc38-0d065042ceca

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-64d475bcc9-z7whx_73595d81-d909-4423-bc38-0d065042ceca became leader

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Created

Container created

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-6f6944bcdd from 1 to 0

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-6f6944bcdd from 0 to 1

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulCreate

Created pod: authorino-6f6944bcdd-7whnd

kuadrant-system

replicaset-controller

authorino-779d94779c

SuccessfulCreate

Created pod: authorino-779d94779c-7l5gf

kuadrant-system

replicaset-controller

authorino-779d94779c

SuccessfulDelete

Deleted pod: authorino-779d94779c-7l5gf

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-64c8bb8959 from 0 to 1

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled up replica set authorino-779d94779c from 0 to 1

kuadrant-system

replicaset-controller

authorino-64c8bb8959

SuccessfulCreate

Created pod: authorino-64c8bb8959-2zlbz

kuadrant-system

replicaset-controller

authorino-6f6944bcdd

SuccessfulDelete

Deleted pod: authorino-6f6944bcdd-7whnd

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-779d94779c from 1 to 0

kuadrant-system

multus

authorino-6f6944bcdd-7whnd

AddedInterface

Add eth0 [10.134.0.44/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-6f6944bcdd-7whnd

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-779d94779c-7l5gf

AddedInterface

Add eth0 [10.134.0.45/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-779d94779c-7l5gf

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-64c8bb8959-2zlbz

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

multus

authorino-64c8bb8959-2zlbz

AddedInterface

Add eth0 [10.134.0.46/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-6f6944bcdd-7whnd

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.374s (1.374s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-779d94779c-7l5gf

Started

Container started

kuadrant-system

kubelet

authorino-6f6944bcdd-7whnd

Created

Container created

kuadrant-system

kubelet

authorino-779d94779c-7l5gf

Created

Container created

kuadrant-system

kubelet

authorino-779d94779c-7l5gf

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.299s (1.299s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-64c8bb8959-2zlbz

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.221s (1.221s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-779d94779c-7l5gf

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6f6944bcdd-7whnd

Killing

Stopping container authorino

kuadrant-system

kubelet

authorino-6f6944bcdd-7whnd

Started

Container started

kuadrant-system

kubelet

authorino-64c8bb8959-2zlbz

Created

Container created

kuadrant-system

kubelet

authorino-64c8bb8959-2zlbz

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

Scaled down replica set authorino-56c6c9ddd5 from 1 to 0
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

ProbeError

Startup probe error: Get "http://10.134.0.23:15021/healthz/ready": dial tcp 10.134.0.23:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Unhealthy

Startup probe failed: Get "http://10.134.0.23:15021/healthz/ready": dial tcp 10.134.0.23:15021: connect: connection refused

kuadrant-system

kubelet

authorino-56c6c9ddd5-ggd22

Killing

Stopping container authorino

kuadrant-system

replicaset-controller

authorino-56c6c9ddd5

SuccessfulDelete

Deleted pod: authorino-56c6c9ddd5-ggd22

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-serving-cert

Generated

Stored new private key in temporary Secret resource "jobset-serving-cert-59hks"

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-metrics-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-metrics-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CertificateCreated

Created Certificate.cert-manager.io/jobset-serving-cert -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

IssuerCreated

Created Issuer.cert-manager.io/jobset-selfsigned-issuer -n openshift-jobset-operator because it was missing

openshift-jobset-operator

cert-manager-certificates-trigger

jobset-serving-cert

Issuing

Issuing certificate as Secret does not exist

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-metrics-cert

Requested

Created new CertificateRequest resource "jobset-metrics-cert-1"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-metrics-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-metrics-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-acme

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

CertificateIssued

Certificate fetched from issuer successfully

openshift-jobset-operator

cert-manager-certificates-request-manager

jobset-serving-cert

Requested

Created new CertificateRequest resource "jobset-serving-cert-1"

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-serving-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificates-key-manager

jobset-metrics-cert

Generated

Stored new private key in temporary Secret resource "jobset-metrics-cert-h6866"

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificates-issuing

jobset-metrics-cert

Issuing

The certificate has been successfully issued

openshift-jobset-operator

cert-manager-certificaterequests-issuer-venafi

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-vault

jobset-metrics-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-approver

jobset-serving-cert-1

cert-manager.io

Certificate request has been approved by cert-manager.io

openshift-jobset-operator

cert-manager-certificaterequests-issuer-selfsigned

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

cert-manager-certificaterequests-issuer-ca

jobset-serving-cert-1

WaitingForApproval

Not signing CertificateRequest until it is Approved

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

CustomResourceDefinitionCreated

Created CustomResourceDefinition.apiextensions.k8s.io/jobsets.jobset.x-k8s.io because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationCreated

Created MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationCreated

Created ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it was missing

openshift-jobset-operator

replicaset-controller

jobset-controller-manager-6ddb96fb5

SuccessfulCreate

Created pod: jobset-controller-manager-6ddb96fb5-b4jlm

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleBindingCreated

Created RoleBinding.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

RoleCreated

Created Role.rbac.authorization.k8s.io/jobset-prometheus-k8s -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ServiceMonitorCreated

Created ServiceMonitor.monitoring.coreos.com/jobset-controller-manager-metrics-monitor -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ConfigMapCreated

Created ConfigMap/jobset-manager-config -n openshift-jobset-operator because it was missing

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

DeploymentCreated

Created Deployment.apps/jobset-controller-manager -n openshift-jobset-operator because it was missing

openshift-jobset-operator

deployment-controller

jobset-controller-manager

ScalingReplicaSet

Scaled up replica set jobset-controller-manager-6ddb96fb5 from 0 to 1

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

MutatingWebhookConfigurationUpdated

Updated MutatingWebhookConfiguration.admissionregistration.k8s.io/jobset-mutating-webhook-configuration because it changed

openshift-jobset-operator

multus

jobset-controller-manager-6ddb96fb5-b4jlm

AddedInterface

Add eth0 [10.133.0.34/23] from ovn-kubernetes

openshift-jobset-operator

openshift-jobset-operator

jobset-operator

ValidatingWebhookConfigurationUpdated

Updated ValidatingWebhookConfiguration.admissionregistration.k8s.io/jobset-validating-webhook-configuration because it changed

openshift-jobset-operator

kubelet

jobset-controller-manager-6ddb96fb5-b4jlm

Pulling

Pulling image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987"

openshift-jobset-operator

kubelet

jobset-controller-manager-6ddb96fb5-b4jlm

Created

Container created

openshift-jobset-operator

kubelet

jobset-controller-manager-6ddb96fb5-b4jlm

Pulled

Successfully pulled image "registry.redhat.io/job-set/jobset-rhel9@sha256:8d2a70fe90b2414cdb269d217703b7e5b3409f9276acf671721b1abc67a0b987" in 3.981s (3.981s including waiting). Image size: 182184678 bytes.

openshift-jobset-operator

kubelet

jobset-controller-manager-6ddb96fb5-b4jlm

Started

Container started

openshift-jobset-operator

jobset-controller-manager-6ddb96fb5-b4jlm_f57126e2-e8b5-4539-a1e1-5c68c96fd267

6d4f6a47.jobset.x-k8s.io

LeaderElection

jobset-controller-manager-6ddb96fb5-b4jlm_f57126e2-e8b5-4539-a1e1-5c68c96fd267 became leader
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled up replica set llmisvc-controller-manager-6466c554d8 from 0 to 1

opendatahub

replicaset-controller

llmisvc-controller-manager-6466c554d8

SuccessfulCreate

Created pod: llmisvc-controller-manager-6466c554d8-446vb

opendatahub

multus

llmisvc-controller-manager-6466c554d8-446vb

AddedInterface

Add eth0 [10.134.0.47/23] from ovn-kubernetes

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-446vb

Pulling

Pulling image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321"

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-446vb

Created

Container created

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-446vb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-kserve-llmisvc-controller@sha256:be3760deb2e1570f182db0819a98b4cebd1de04de25b8c9bdd0b38396f0be321" in 2.278s (2.278s including waiting). Image size: 235100981 bytes.

opendatahub

kubelet

llmisvc-controller-manager-6466c554d8-446vb

Started

Container started

opendatahub

deployment-controller

llmisvc-controller-manager

ScalingReplicaSet

Scaled down replica set llmisvc-controller-manager-869d5dcc5d from 1 to 0

opendatahub

kubelet

llmisvc-controller-manager-869d5dcc5d-82cfd

Killing

Stopping container manager

opendatahub

replicaset-controller

llmisvc-controller-manager-869d5dcc5d

SuccessfulDelete

Deleted pod: llmisvc-controller-manager-869d5dcc5d-82cfd

opendatahub

replicaset-controller

ai-gateway-controller-7d85675f98

SuccessfulCreate

Created pod: ai-gateway-controller-7d85675f98-87wcp

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-7d85675f98 from 0 to 1

opendatahub

multus

ai-gateway-controller-7d85675f98-87wcp

AddedInterface

Add eth0 [10.132.0.33/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-7d85675f98-87wcp

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

kubelet

ai-gateway-controller-7d85675f98-87wcp

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 1.781s (1.781s including waiting). Image size: 183979433 bytes.

opendatahub

kubelet

ai-gateway-controller-7d85675f98-87wcp

Started

Container started

opendatahub

kubelet

ai-gateway-controller-7d85675f98-87wcp

Created

Container created

opendatahub

ai-gateway-controller-7d85675f98-87wcp_72569f70-0760-467f-aff1-4529847e6af5

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-7d85675f98-87wcp_72569f70-0760-467f-aff1-4529847e6af5 became leader
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

FailedCreate

Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

FailedCreate

Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulCreate

Created pod: payload-pre-processing-7b98bb678c-7dngs

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulCreate

Created pod: payload-processing-55fbbbf9dc-q2fb6

openshift-ingress

multus

payload-processing-55fbbbf9dc-q2fb6

AddedInterface

Add eth0 [10.132.0.35/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7dngs

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-q2fb6

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-7b98bb678c-7dngs

AddedInterface

Add eth0 [10.132.0.34/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7dngs

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.676s (2.676s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7dngs

Created

Container created

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-q2fb6

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.841s (2.841s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-q2fb6

Created

Container created

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-q2fb6

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7dngs

Started

Container started

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled up replica set ai-gateway-controller-575f85f67b from 0 to 1

opendatahub

replicaset-controller

ai-gateway-controller-575f85f67b

SuccessfulCreate

Created pod: ai-gateway-controller-575f85f67b-6vdks

opendatahub

multus

ai-gateway-controller-575f85f67b-6vdks

AddedInterface

Add eth0 [10.132.0.36/23] from ovn-kubernetes

opendatahub

kubelet

ai-gateway-controller-575f85f67b-6vdks

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-controller:latest"

opendatahub

llmisvc-controller-manager-6466c554d8-446vb_76a8bea2-cbc0-4551-97a3-4211cfe14302

llminferenceservice-kserve-controller-manager

LeaderElection

llmisvc-controller-manager-6466c554d8-446vb_76a8bea2-cbc0-4551-97a3-4211cfe14302 became leader

opendatahub

kubelet

ai-gateway-controller-575f85f67b-6vdks

Created

Container created

opendatahub

kubelet

ai-gateway-controller-575f85f67b-6vdks

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-controller:latest" in 844ms (844ms including waiting). Image size: 183979433 bytes.

opendatahub

kubelet

ai-gateway-controller-575f85f67b-6vdks

Started

Container started
(x2)

opendatahub

kubelet

maas-controller-64d475bcc9-z7whx

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159"

opendatahub

kubelet

maas-controller-64d475bcc9-z7whx

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159" in 850ms (850ms including waiting). Image size: 200094742 bytes.
(x2)

opendatahub

kubelet

maas-controller-64d475bcc9-z7whx

Created

Container created
(x2)

opendatahub

kubelet

maas-controller-64d475bcc9-z7whx

Started

Container started

opendatahub

replicaset-controller

ai-gateway-controller-7d85675f98

SuccessfulDelete

Deleted pod: ai-gateway-controller-7d85675f98-87wcp

opendatahub

deployment-controller

ai-gateway-controller

ScalingReplicaSet

Scaled down replica set ai-gateway-controller-7d85675f98 from 1 to 0

opendatahub

kubelet

ai-gateway-controller-7d85675f98-87wcp

Killing

Stopping container manager

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

opendatahub

maas-controller-64d475bcc9-z7whx_3b78b46c-3ec8-4ea4-903b-5bbaf49e09ad

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-64d475bcc9-z7whx_3b78b46c-3ec8-4ea4-903b-5bbaf49e09ad became leader

llm

deployment-controller

e2e-distinct-2-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-2-simulated-kserve-cbd4f5586 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-2-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Service llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.Deployment llm/e2e-distinct-2-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-2-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-distinct-2-simulated-kserve-cbd4f5586

SuccessfulCreate

Created pod: e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-2-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-2-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

AddedInterface

Add eth0 [10.132.0.37/23] from ovn-kubernetes

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 2.041s (2.041s including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

Created

Container created

llm

kubelet

e2e-distinct-2-simulated-kserve-cbd4f5586-znq7h

Started

Container started

llm

deployment-controller

e2e-distinct-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-distinct-simulated-kserve-7b998b79d9 from 0 to 1

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.DestinationRule llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.HTTPRoute llm/e2e-distinct-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Service llm/e2e-distinct-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Deployment llm/e2e-distinct-simulated-kserve

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

replicaset-controller

e2e-distinct-simulated-kserve-7b998b79d9

SuccessfulCreate

Created pod: e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.NetworkPolicy llm/e2e-distinct-simulated-prometheus-scraping

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

Started

Container started

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

Created

Container created

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 574ms (574ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-distinct-simulated-kserve-7b998b79d9-gzgw9

AddedInterface

Add eth0 [10.132.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-distinct-simulated

Created

Created v1.PodMonitor llm/e2e-distinct-simulated-kserve-llmisvc-engine

opendatahub

ai-gateway-controller-575f85f67b-6vdks_b8071925-63f8-4cc1-8e3e-afb361da7be3

ai-gateway-controller-leader.opendatahub.io

LeaderElection

ai-gateway-controller-575f85f67b-6vdks_b8071925-63f8-4cc1-8e3e-afb361da7be3 became leader

llm

deployment-controller

e2e-embedding-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-embedding-simulated-kserve-67bf594cd6 from 0 to 1

llm

replicaset-controller

e2e-embedding-simulated-kserve-67bf594cd6

SuccessfulCreate

Created pod: e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Service llm/e2e-embedding-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Deployment llm/e2e-embedding-simulated-kserve

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.DestinationRule llm/e2e-embedding-simulated-kserve-workload-svc

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

AddedInterface

Add eth0 [10.132.0.39/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.PodMonitor llm/e2e-embedding-simulated-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.HTTPRoute llm/e2e-embedding-simulated-kserve-route

llm

LLMInferenceServiceController

e2e-embedding-simulated

Created

Created v1.NetworkPolicy llm/e2e-embedding-simulated-prometheus-scraping

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 577ms (577ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

Created

Container created

llm

kubelet

e2e-embedding-simulated-kserve-67bf594cd6-pmrj9

Started

Container started

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.DestinationRule llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

replicaset-controller

e2e-trlp-test-simulated-kserve-69b588897d

SuccessfulCreate

Created pod: e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

llm

deployment-controller

e2e-trlp-test-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-trlp-test-simulated-kserve-69b588897d from 0 to 1

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Deployment llm/e2e-trlp-test-simulated-kserve

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.Service llm/e2e-trlp-test-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.HTTPRoute llm/e2e-trlp-test-simulated-kserve-route

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine-default

llm

multus

e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

AddedInterface

Add eth0 [10.133.0.35/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.NetworkPolicy llm/e2e-trlp-test-simulated-prometheus-scraping

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Created

Created v1.PodMonitor llm/e2e-trlp-test-simulated-kserve-llmisvc-engine

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

Started

Container started

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

Created

Container created

llm

kubelet

e2e-trlp-test-simulated-kserve-69b588897d-gh8jf

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 1.474s (1.474s including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

replicaset-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc

SuccessfulCreate

Created pod: e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

llm

deployment-controller

e2e-unconfigured-facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8f4fc from 0 to 1

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Deployment llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.Service llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-route

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 565ms (565ms including waiting). Image size: 98346788 bytes.

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

Started

Container started

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/e2e-unconab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/e2e-unconfiab60ef4d3a239b5143b412cab04acac3-prometheus-scraping

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/eab60ef4d3a239b5143b412cab04acac3-kserve-llmisvc-engine-default

llm

multus

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

AddedInterface

Add eth0 [10.132.0.40/23] from ovn-kubernetes

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

kubelet

e2e-unconfigured-facebook-opt-125m-simulated-kserve-7658b8hjb2f

Created

Container created

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Secret llm/echo-simulator-kserve-self-signed-certs

llm

replicaset-controller

echo-simulator-kserve-64b6f8f96c

SuccessfulCreate

Created pod: echo-simulator-kserve-64b6f8f96c-5bs7b

llm

deployment-controller

echo-simulator-kserve

ScalingReplicaSet

Scaled up replica set echo-simulator-kserve-64b6f8f96c from 0 to 1

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Deployment llm/echo-simulator-kserve

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.DestinationRule llm/echo-simulator-kserve-workload-svc

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.HTTPRoute llm/echo-simulator-kserve-route

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.Service llm/echo-simulator-kserve-workload-svc

llm

kubelet

echo-simulator-kserve-64b6f8f96c-5bs7b

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0"

llm

multus

echo-simulator-kserve-64b6f8f96c-5bs7b

AddedInterface

Add eth0 [10.133.0.36/23] from ovn-kubernetes

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.NetworkPolicy llm/echo-simulator-prometheus-scraping

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine

llm

LLMInferenceServiceController

echo-simulator

Created

Created v1.PodMonitor llm/echo-simulator-kserve-llmisvc-engine-default

llm

kubelet

echo-simulator-kserve-64b6f8f96c-5bs7b

Created

Container created

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Deployment llm/facebook-opt-125m-simulated-kserve

llm

kubelet

echo-simulator-kserve-64b6f8f96c-5bs7b

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.9.0" in 2.489s (2.489s including waiting). Image size: 78208878 bytes.

llm

kubelet

echo-simulator-kserve-64b6f8f96c-5bs7b

Started

Container started

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Service llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

deployment-controller

facebook-opt-125m-simulated-kserve

ScalingReplicaSet

Scaled up replica set facebook-opt-125m-simulated-kserve-84b4ff656c from 0 to 1

llm

replicaset-controller

facebook-opt-125m-simulated-kserve-84b4ff656c

SuccessfulCreate

Created pod: facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.HTTPRoute llm/facebook-opt-125m-simulated-kserve-route

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.DestinationRule llm/facebook-opt-125m-simulated-kserve-workload-svc

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.PodMonitor llm/facebook-opt-125m-simulated-kserve-llmisvc-engine-default

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"

llm

multus

facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

AddedInterface

Add eth0 [10.133.0.37/23] from ovn-kubernetes

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

Created

Container created

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 597ms (597ms including waiting). Image size: 98346788 bytes.

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Created

Created v1.NetworkPolicy llm/facebook-opt-125m-simulated-prometheus-scraping

llm

kubelet

facebook-opt-125m-simulated-kserve-84b4ff656c-6r8gx

Started

Container started

llm

replicaset-controller

premium-simulated-simulated-premium-kserve-5f5c4b459f

SuccessfulCreate

Created pod: premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.HTTPRoute llm/premium-simulated-simulated-premium-kserve-route

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Service llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.DestinationRule llm/premium-simulated-simulated-premium-kserve-workload-svc

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.Deployment llm/premium-simulated-simulated-premium-kserve

llm

deployment-controller

premium-simulated-simulated-premium-kserve

ScalingReplicaSet

Scaled up replica set premium-simulated-simulated-premium-kserve-5f5c4b459f from 0 to 1

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/p75d89895ad19b4ac62612d08bfe7f574-kserve-llmisvc-engine-default

llm

multus

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

AddedInterface

Add eth0 [10.133.0.38/23] from ovn-kubernetes

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.NetworkPolicy llm/premium-simulated-simulated-premium-prometheus-scraping

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Created

Created v1.PodMonitor llm/premium-simulated-simulated-premium-kserve-llmisvc-engine

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Started

Container started

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Created

Container created

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Pulled

Successfully pulled image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2" in 557ms (557ms including waiting). Image size: 98346788 bytes.

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Pulling

Pulling image "ghcr.io/llm-d/llm-d-inference-sim:v0.8.2"
(x2)

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

Updated

Updated v1.Secret llm/e2e-distinct-2-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-2-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-2-simulated] is Ready

llm

LLMInferenceServiceController

e2e-distinct-simulated

Updated

Updated v1.Secret llm/e2e-distinct-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-distinct-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-distinct-simulated] is Ready

llm

LLMInferenceServiceController

e2e-embedding-simulated

Updated

Updated v1.Secret llm/e2e-embedding-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-embedding-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-embedding-simulated] is Ready

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

Updated

Updated v1.Secret llm/e2e-trlp-test-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-trlp-test-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-trlp-test-simulated] is Ready

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-unconfigured-facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [e2e-unconfigured-facebook-opt-125m-simulated] is Ready

llm

LLMInferenceServiceController

echo-simulator

Updated

Updated v1.Secret llm/echo-simulator-kserve-self-signed-certs

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-g2829

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 1 to 4

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

AddedInterface

Add eth0 [10.132.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

AddedInterface

Add eth0 [10.132.0.41/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-g2829

AddedInterface

Add eth0 [10.133.0.39/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-x4lgh

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-qpmzx

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55"

llm

LLMInferenceServiceController

echo-simulator

LLMInferenceServiceReady

LLMInferenceService [echo-simulator] is Ready

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

Updated

Updated v1.Secret llm/facebook-opt-125m-simulated-kserve-self-signed-certs

llm

LLMInferenceServiceController

facebook-opt-125m-simulated

LLMInferenceServiceReady

LLMInferenceService [facebook-opt-125m-simulated] is Ready

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.816s (3.816s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.572s (3.572s including waiting). Image size: 183555333 bytes.

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.755s (3.755s including waiting). Image size: 183555333 bytes.

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

Updated

Updated v1.Secret llm/premium-simulated-simulated-premium-kserve-self-signed-certs

llm

LLMInferenceServiceController

premium-simulated-simulated-premium

LLMInferenceServiceReady

LLMInferenceService [premium-simulated-simulated-premium] is Ready

kuadrant-system

replicaset-controller

authorino-58588b67c6

SuccessfulCreate

Created pod: authorino-58588b67c6-mr9nn

kuadrant-system

multus

authorino-58588b67c6-mr9nn

AddedInterface

Add eth0 [10.134.0.48/23] from ovn-kubernetes

kuadrant-system

kubelet

authorino-58588b67c6-mr9nn

Pulling

Pulling image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0"

kuadrant-system

kubelet

authorino-58588b67c6-mr9nn

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

ProbeError

Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Unhealthy

Startup probe failed: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

ProbeError

Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused body:

kuadrant-system

kubelet

authorino-58588b67c6-mr9nn

Created

Container created
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Unhealthy

Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

ProbeError

Startup probe error: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Unhealthy

Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused

kuadrant-system

kubelet

authorino-58588b67c6-mr9nn

Pulled

Successfully pulled image "registry.redhat.io/rhcl-1/authorino-rhel9@sha256:c4a5a8f911a5097293de58dde320e9b8e51783b8fd92121de73bc60f6672cdf0" in 1.578s (1.578s including waiting). Image size: 204096810 bytes.

kuadrant-system

kubelet

authorino-64c8bb8959-2zlbz

Killing

Stopping container authorino
(x2)

kuadrant-system

deployment-controller

authorino

ScalingReplicaSet

(combined from similar events): Scaled down replica set authorino-64c8bb8959 from 1 to 0

kuadrant-system

replicaset-controller

authorino-64c8bb8959

SuccessfulDelete

Deleted pod: authorino-64c8bb8959-2zlbz

openshift-ingress

gateway_labeler_controller

e2e-worker-w2-8ff537

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-8ff537

openshift-ingress

gateway_labeler_controller

e2e-worker-w0-3ba2f3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-3ba2f3

openshift-ingress

gateway_labeler_controller

e2e-worker-w4-190724

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-190724

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 0 to 1

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

AddedInterface

Add eth0 [10.132.0.43/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 0 to 1

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

AddedInterface

Add eth0 [10.133.0.40/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

AddedInterface

Add eth0 [10.132.0.44/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w2-8ff537

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-8ff537-5d69f665cc

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w2-8ff537

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w2-8ff537-5d69f665cc from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w2-8ff537-5d69f665cc

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w2-8ff537-5d69f665cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-8ff537: serviceaccount "payload-processing-e2e-worker-w2-8ff537" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-8ff537: serviceaccount "payload-processing-e2e-worker-w2-8ff537" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

AddedInterface

Add eth0 [10.132.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

AddedInterface

Add eth0 [10.132.0.45/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

Started

Container started

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w4-190724

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w4-190724-65b8997649 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-190724-8bb994975

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w4-190724-8bb994975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-190724: serviceaccount "payload-processing-e2e-worker-w4-190724" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w4-190724

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w4-190724-8bb994975 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w4-190724-8bb994975

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-190724-65b8997649

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w4-190724-65b8997649

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w4-190724-65b8997649-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-190724: serviceaccount "payload-processing-e2e-worker-w4-190724" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

AddedInterface

Add eth0 [10.133.0.42/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

AddedInterface

Add eth0 [10.133.0.41/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-3ba2f3: serviceaccount "payload-processing-e2e-worker-w0-3ba2f3" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-3ba2f3: serviceaccount "payload-processing-e2e-worker-w0-3ba2f3" not found

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w0-3ba2f3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w0-3ba2f3-5b69fb897 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w0-3ba2f3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Pulling

Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable"

openshift-ingress

multus

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

AddedInterface

Add eth0 [10.134.0.49/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

openshift-ingress

multus

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

AddedInterface

Add eth0 [10.133.0.43/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.821s (2.821s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 802ms (802ms including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.901s (2.901s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Pulled

Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.526s (2.526s including waiting). Image size: 125353963 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

Unhealthy

Startup probe failed: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

ProbeError

Startup probe error: Get "http://10.132.0.44:15021/healthz/ready": dial tcp 10.132.0.44:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Unhealthy

Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

ProbeError

Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Unhealthy

Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Unhealthy

Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

ProbeError

Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

ProbeError

Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused body:

openshift-ingress

gateway_labeler_controller

e2e-inf-a-85584b

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-85584b

openshift-ingress

multus

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

AddedInterface

Add eth0 [10.133.0.45/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-inf-a-85584b-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-a-85584b-openshift-default-879d4df9f from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-a-85584b-openshift-default-879d4df9f

SuccessfulCreate

Created pod: e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-a-85584b

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-inf-a-85584b

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-a-85584b-6f56f6ff49 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-85584b-6f56f6ff49

FailedCreate

Error creating: pods "payload-processing-e2e-inf-a-85584b-6f56f6ff49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-85584b: serviceaccount "payload-processing-e2e-inf-a-85584b" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-85584b: serviceaccount "payload-processing-e2e-inf-a-85584b" not found

openshift-ingress

multus

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

AddedInterface

Add eth0 [10.134.0.51/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-a-85584b-6f56f6ff49

SuccessfulCreate

Created pod: payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

openshift-ingress

kubelet

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

Started

Container started

openshift-ingress

multus

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

AddedInterface

Add eth0 [10.134.0.52/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-r5wn2

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-g6cln

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 4 to 8

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

AddedInterface

Add eth0 [10.132.0.50/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-nrzv7

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-wvfkd

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

AddedInterface

Add eth0 [10.134.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-g6cln_openshift-ingress_8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a_0(c6dfe699a2568e3f1461c011f2e1a8a7907be568bffa813a45776186e9c913ed): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-g6cln to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c6dfe699a2568e3f1461c011f2e1a8a7907be568bffa813a45776186e9c913ed" Netns:"/var/run/netns/0302f7d5-0f56-417e-b291-b36e61be63bc" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-g6cln;K8S_POD_INFRA_CONTAINER_ID=c6dfe699a2568e3f1461c011f2e1a8a7907be568bffa813a45776186e9c913ed;K8S_POD_UID=8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln/8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-g6cln?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-nrzv7_openshift-ingress_83242ea6-ab3a-4ac0-a37e-012b6c0b4265_0(8b278b9bce9ae0ab27a05466bb8b9ce1d471bb4c55e988595a83439b3b82cf0e): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-nrzv7 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"8b278b9bce9ae0ab27a05466bb8b9ce1d471bb4c55e988595a83439b3b82cf0e" Netns:"/var/run/netns/1ef4a2c8-00c7-405a-a254-e5072777328e" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-nrzv7;K8S_POD_INFRA_CONTAINER_ID=8b278b9bce9ae0ab27a05466bb8b9ce1d471bb4c55e988595a83439b3b82cf0e;K8S_POD_UID=83242ea6-ab3a-4ac0-a37e-012b6c0b4265" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7/83242ea6-ab3a-4ac0-a37e-012b6c0b4265]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-nrzv7?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-g6cln_openshift-ingress_8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a_0(3e138a7829c55026afd9aae87527a835bfbd9d12ac0911101a881ef254aaa029): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-g6cln to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"3e138a7829c55026afd9aae87527a835bfbd9d12ac0911101a881ef254aaa029" Netns:"/var/run/netns/af072dc1-7674-4037-9207-828ff7ec00f3" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-g6cln;K8S_POD_INFRA_CONTAINER_ID=3e138a7829c55026afd9aae87527a835bfbd9d12ac0911101a881ef254aaa029;K8S_POD_UID=8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln/8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-g6cln?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-nrzv7_openshift-ingress_83242ea6-ab3a-4ac0-a37e-012b6c0b4265_0(1f9f369da12d30ceb8322c3d6d4efcd380f33f92682339c34ceeb0f7b5e9be04): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-nrzv7 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"1f9f369da12d30ceb8322c3d6d4efcd380f33f92682339c34ceeb0f7b5e9be04" Netns:"/var/run/netns/922c9c66-1f17-4981-b566-7ec272a0c922" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-nrzv7;K8S_POD_INFRA_CONTAINER_ID=1f9f369da12d30ceb8322c3d6d4efcd380f33f92682339c34ceeb0f7b5e9be04;K8S_POD_UID=83242ea6-ab3a-4ac0-a37e-012b6c0b4265" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7/83242ea6-ab3a-4ac0-a37e-012b6c0b4265]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-nrzv7?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

ProbeError

Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

Unhealthy

Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Unhealthy

Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

ProbeError

Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-kvv59

AddedInterface

Add eth0 [10.134.0.56/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-kvv59

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason:

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

(combined from similar events): Created pod: maas-default-gateway-openshift-default-7d9897b54c-6zskg

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-kvv59

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-kvv59

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-kvv59

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-6zskg_openshift-ingress_db5ba475-fdb0-417a-8799-d8448ad561dc_0(c57356a18c1053d701d00be65e0f2ae26051049532904f11903b70f0858cee38): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-6zskg to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c57356a18c1053d701d00be65e0f2ae26051049532904f11903b70f0858cee38" Netns:"/var/run/netns/e80589aa-88ac-4da2-91e6-64e2ad82cea5" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-6zskg;K8S_POD_INFRA_CONTAINER_ID=c57356a18c1053d701d00be65e0f2ae26051049532904f11903b70f0858cee38;K8S_POD_UID=db5ba475-fdb0-417a-8799-d8448ad561dc" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-6zskg] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-6zskg/db5ba475-fdb0-417a-8799-d8448ad561dc]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-6zskg?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-nrzv7_openshift-ingress_83242ea6-ab3a-4ac0-a37e-012b6c0b4265_0(015cfd17b3b4dc2bb859b9bbafc643986decccdda71085b005711f5284910c99): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-nrzv7 to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"015cfd17b3b4dc2bb859b9bbafc643986decccdda71085b005711f5284910c99" Netns:"/var/run/netns/ea3e98a6-e61c-449f-a33c-bc533b332152" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-nrzv7;K8S_POD_INFRA_CONTAINER_ID=015cfd17b3b4dc2bb859b9bbafc643986decccdda71085b005711f5284910c99;K8S_POD_UID=83242ea6-ab3a-4ac0-a37e-012b6c0b4265" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-nrzv7/83242ea6-ab3a-4ac0-a37e-012b6c0b4265]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-nrzv7?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-6zskg_openshift-ingress_db5ba475-fdb0-417a-8799-d8448ad561dc_0(c2233bf2595d00f82f1b6731d8ff1244b240ca9cd07cdea434f3f8656fcfeeaf): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-6zskg to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"c2233bf2595d00f82f1b6731d8ff1244b240ca9cd07cdea434f3f8656fcfeeaf" Netns:"/var/run/netns/bd8d0746-44d5-432f-b6e5-f4d645b557e6" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-6zskg;K8S_POD_INFRA_CONTAINER_ID=c2233bf2595d00f82f1b6731d8ff1244b240ca9cd07cdea434f3f8656fcfeeaf;K8S_POD_UID=db5ba475-fdb0-417a-8799-d8448ad561dc" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-6zskg] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-6zskg/db5ba475-fdb0-417a-8799-d8448ad561dc]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-6zskg?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

FailedCreatePodSandBox

Failed to create pod sandbox: rpc error: code = Unknown desc = failed to create pod network sandbox k8s_maas-default-gateway-openshift-default-7d9897b54c-g6cln_openshift-ingress_8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a_0(1c85a7f05e6d6c55dc3cbd3f1033c0606f1590430eda2e01b0c106861eb345a0): error adding pod openshift-ingress_maas-default-gateway-openshift-default-7d9897b54c-g6cln to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"1c85a7f05e6d6c55dc3cbd3f1033c0606f1590430eda2e01b0c106861eb345a0" Netns:"/var/run/netns/f17d5b3d-3d2a-42f2-8840-4c93c4743b21" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=openshift-ingress;K8S_POD_NAME=maas-default-gateway-openshift-default-7d9897b54c-g6cln;K8S_POD_INFRA_CONTAINER_ID=1c85a7f05e6d6c55dc3cbd3f1033c0606f1590430eda2e01b0c106861eb345a0;K8S_POD_UID=8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a" Path:"" ERRORED: error configuring pod [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln] networking: Multus: [openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-g6cln/8b1fdc09-8d2b-4c4e-bb72-3c4e15342f8a]: error waiting for pod: Get "https://ae23e5d867b4645df8e862367584a87b-ec09ebbe119c1d65.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/openshift-ingress/pods/maas-default-gateway-openshift-default-7d9897b54c-g6cln?timeout=1m0s": context deadline exceeded ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-kvv59

Unhealthy

Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-kvv59

ProbeError

Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Unhealthy

Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

ProbeError

Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-85584b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-85584b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-6zskg

AddedInterface

Add eth0 [10.133.0.48/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

AddedInterface

Add eth0 [10.133.0.46/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

e2e-inf-b-ef9b55-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-inf-b-ef9b55-openshift-default-6ccbb7664f from 0 to 1

openshift-ingress

replicaset-controller

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f

SuccessfulCreate

Created pod: e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

openshift-ingress

gateway_labeler_controller

e2e-inf-b-ef9b55

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ef9b55

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-g6cln

AddedInterface

Add eth0 [10.133.0.47/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

AddedInterface

Add eth0 [10.132.0.52/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-85584b-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-a-85584b-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

Unhealthy

Startup probe failed: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

Unhealthy

Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

ProbeError

Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

ProbeError

Startup probe error: Get "http://10.133.0.48:15021/healthz/ready": dial tcp 10.133.0.48:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

ProbeError

Startup probe error: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

Unhealthy

Startup probe failed: Get "http://10.133.0.47:15021/healthz/ready": dial tcp 10.133.0.47:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-inf-b-ef9b55

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-inf-b-ef9b55-785fcbf955 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79

SuccessfulCreate

Created pod: payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79

FailedCreate

Error creating: pods "payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ef9b55: serviceaccount "payload-processing-e2e-inf-b-ef9b55" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-ef9b55-785fcbf955

SuccessfulCreate

Created pod: payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-inf-b-ef9b55-785fcbf955

FailedCreate

Error creating: pods "payload-processing-e2e-inf-b-ef9b55-785fcbf955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ef9b55: serviceaccount "payload-processing-e2e-inf-b-ef9b55" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-inf-b-ef9b55

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

Started

Container started

openshift-ingress

multus

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

AddedInterface

Add eth0 [10.134.0.58/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

AddedInterface

Add eth0 [10.132.0.53/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7689-6a6333

AddedInterface

Add eth0 [10.133.0.50/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7689-6a6333

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

kubelet

test-curl-7689-6a6333

Created

Container created

opendatahub

kubelet

test-curl-7689-6a6333

Started

Container started

opendatahub

kubelet

test-curl-7689-6a6333

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.587s (2.587s including waiting). Image size: 106100986 bytes.

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

ProbeError

Readiness probe error: Get "http://10.132.0.52:15021/healthz/ready": read tcp 10.132.0.2:57606->10.132.0.52:15021: read: connection reset by peer body:

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Unhealthy

Readiness probe failed: Get "http://10.132.0.52:15021/healthz/ready": read tcp 10.132.0.2:57606->10.132.0.52:15021: read: connection reset by peer
(x2)

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7689-899601

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7689-6a6333

Killing

Stopping container test-curl-7689-6a6333

opendatahub

kubelet

test-curl-7689-899601

Started

Container started

opendatahub

kubelet

test-curl-7689-899601

Created

Container created

opendatahub

multus

test-curl-7689-899601

AddedInterface

Add eth0 [10.133.0.52/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7689-899601

Killing

Stopping container test-curl-7689-899601

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-c79abe

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-c79abe
(x6)

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

ProbeError

Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Unhealthy

Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused

openshift-ingress

deployment-controller

e2e-extmod-a-c79abe-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-c79abe-openshift-default-94746d989 from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-a-c79abe-openshift-default-94746d989

SuccessfulCreate

Created pod: e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

openshift-ingress

multus

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

AddedInterface

Add eth0 [10.133.0.53/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-c79abe: serviceaccount "payload-processing-e2e-extmod-a-c79abe" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-c79abe

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-c79abe: serviceaccount "payload-processing-e2e-extmod-a-c79abe" not found

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-c79abe

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-c79abe-5dc676ccd5 from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

AddedInterface

Add eth0 [10.132.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

Created

Container created

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

openshift-ingress

multus

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

AddedInterface

Add eth0 [10.134.0.60/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

Started

Container started
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-ef9b55-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-inf-b-ef9b55-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Created

Container created

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

AddedInterface

Add eth0 [10.133.0.55/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

AddedInterface

Add eth0 [10.132.0.56/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

ProbeError

Startup probe error: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Unhealthy

Startup probe failed: Get "http://10.133.0.55:15021/healthz/ready": dial tcp 10.133.0.55:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

Unhealthy

Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

ProbeError

Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c79abe-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c79abe-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

kubelet

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

ProbeError

Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused body:
(x4)

openshift-ingress

kubelet

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

Unhealthy

Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-dc6e48

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-dc6e48

openshift-ingress

deployment-controller

e2e-extmod-b-dc6e48-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc from 0 to 1

openshift-ingress

multus

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

AddedInterface

Add eth0 [10.133.0.59/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc

SuccessfulCreate

Created pod: e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-dc6e48

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-dc6e48-6869d88cd5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 2 to 3

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-dc6e48: serviceaccount "payload-processing-e2e-extmod-b-dc6e48" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-dc6e48

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Created

Container created
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-dc6e48: serviceaccount "payload-processing-e2e-extmod-b-dc6e48" not found

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

AddedInterface

Add eth0 [10.134.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

AddedInterface

Add eth0 [10.132.0.59/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

AddedInterface

Add eth0 [10.132.0.58/23] from ovn-kubernetes
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c79abe-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-c79abe-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Unhealthy

Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

ProbeError

Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-dc6e48-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-dc6e48-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 3 to 4

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

AddedInterface

Add eth0 [10.134.0.65/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Created

Container created
(x7)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Unhealthy

Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

ProbeError

Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused body:

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Started

Container started

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

AddedInterface

Add eth0 [10.133.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Unhealthy

Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Unhealthy

Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

ProbeError

Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

ProbeError

Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-ipp-a-bd9d24-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-a-bd9d24-openshift-default-6774c4b897 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ipp-a-bd9d24

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-bd9d24

openshift-ingress

replicaset-controller

e2e-ipp-a-bd9d24-openshift-default-6774c4b897

SuccessfulCreate

Created pod: e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

openshift-ingress

multus

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

AddedInterface

Add eth0 [10.133.0.70/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-dc6e48-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-dc6e48-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 2 to 3

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 4 to 7

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

Started

Container started

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

AddedInterface

Add eth0 [10.134.0.70/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-bd9d24

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-bd9d24-56cd9d9699 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-bd9d24

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd9d24: serviceaccount "payload-processing-e2e-ipp-a-bd9d24" not found

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd9d24: serviceaccount "payload-processing-e2e-ipp-a-bd9d24" not found

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

AddedInterface

Add eth0 [10.132.0.63/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw

AddedInterface

Add eth0 [10.132.0.64/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

AddedInterface

Add eth0 [10.134.0.71/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

AddedInterface

Add eth0 [10.132.0.62/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

AddedInterface

Add eth0 [10.133.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Created

Container created
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Unhealthy

Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

ProbeError

Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

ProbeError

Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

ProbeError

Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

ProbeError

Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

Unhealthy

Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

Unhealthy

Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

Unhealthy

Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 3; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 2 to 3

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

AddedInterface

Add eth0 [10.134.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g6cln

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-nrzv7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-g6cln

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-nrzv7
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

ProbeError

Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Unhealthy

Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 7 to 10

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Started

Container started

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

AddedInterface

Add eth0 [10.132.0.67/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Created

Container created

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

AddedInterface

Add eth0 [10.133.0.78/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Created

Container created

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

AddedInterface

Add eth0 [10.133.0.77/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Created

Container created

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r5wn2

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-wvfkd

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-r5wn2

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-wvfkd

Killing

Stopping container istio-proxy
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

ProbeError

Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Unhealthy

Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

ProbeError

Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

ProbeError

Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Unhealthy

Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Unhealthy

Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-bd9d24-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-bd9d24-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

Unhealthy

Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

ProbeError

Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused body:

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 3 to 5

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

AddedInterface

Add eth0 [10.133.0.80/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Started

Container started

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

AddedInterface

Add eth0 [10.134.0.79/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

ProbeError

Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Unhealthy

Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

ProbeError

Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Unhealthy

Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-56cd9d9699-vp9lw

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45

SuccessfulCreate

Created pod: e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-85ff89f5-l5ddq

Killing

Stopping container payload-pre-processing

openshift-ingress

gateway_labeler_controller

e2e-ipp-b-b354e3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-b354e3

openshift-ingress

deployment-controller

e2e-ipp-b-b354e3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ipp-b-b354e3-openshift-default-fdb8d8f45 from 0 to 1

openshift-ingress

multus

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

AddedInterface

Add eth0 [10.132.0.68/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-b354e3-cbf5c5747

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-b354e3-cbf5c5747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b354e3: serviceaccount "payload-processing-e2e-ipp-b-b354e3" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-b354e3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-b354e3-cbf5c5747 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-b354e3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b354e3: serviceaccount "payload-processing-e2e-ipp-b-b354e3" not found

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

AddedInterface

Add eth0 [10.132.0.70/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-b354e3-cbf5c5747

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

AddedInterface

Add eth0 [10.132.0.69/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-bd9d24-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-a-bd9d24-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-b354e3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-b354e3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-a-85584b-5ccf7b6dff-d8ldx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-a-85584b-6f56f6ff49-ls466

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-a-bd9d24

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-a-bd9d24

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87 from 0 to 1

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-inf-a-85584b-openshift-default-879d4df9f-6qm5s

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x8)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd9d24: serviceaccount "payload-processing-e2e-ipp-a-bd9d24" not found
(x9)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-bd9d24: serviceaccount "payload-processing-e2e-ipp-a-bd9d24" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

openshift-ingress

multus

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

AddedInterface

Add eth0 [10.133.0.83/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 8; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 5 to 8

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

AddedInterface

Add eth0 [10.133.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

Pulling

Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable"

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

AddedInterface

Add eth0 [10.132.0.72/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

Started

Container started

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

AddedInterface

Add eth0 [10.133.0.85/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

AddedInterface

Add eth0 [10.132.0.71/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.207s (3.207s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

Pulled

Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.792s (2.792s including waiting). Image size: 166620835 bytes.

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

ProbeError

Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

Unhealthy

Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

ProbeError

Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

ProbeError

Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

Unhealthy

Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Unhealthy

Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": net/http: request canceled (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

payload-processing-e2e-inf-b-ef9b55-785fcbf955-hwknx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-inf-b-ef9b55-7bdbc7ff79-p5k8h

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-inf-b-ef9b55-openshift-default-6ccbb7664f-h67kw

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-praxis-a-e7b2a3-openshift-default-578969456f

SuccessfulCreate

Created pod: e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

openshift-ingress

deployment-controller

e2e-praxis-a-e7b2a3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-a-e7b2a3-openshift-default-578969456f from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-praxis-a-e7b2a3

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-a-e7b2a3

openshift-ingress

multus

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

AddedInterface

Add eth0 [10.133.0.87/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-a-e7b2a3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-a-e7b2a3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-a-e7b2a3-ffb99b799 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-e7b2a3: serviceaccount "payload-processing-e2e-praxis-a-e7b2a3" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-a-e7b2a3: serviceaccount "payload-processing-e2e-praxis-a-e7b2a3" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

Created

Container created

openshift-ingress

multus

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

AddedInterface

Add eth0 [10.132.0.74/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

AddedInterface

Add eth0 [10.134.0.83/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Unhealthy

Readiness probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

ProbeError

Readiness probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-e7b2a3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-e7b2a3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 8 to 10

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

(combined from similar events): Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Created

Container created

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

Started

Container started

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

AddedInterface

Add eth0 [10.132.0.75/23] from ovn-kubernetes

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

AddedInterface

Add eth0 [10.134.0.86/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Unhealthy

Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

ProbeError

Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

ProbeError

Startup probe error: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

Unhealthy

Startup probe failed: Get "http://10.134.0.86:15021/healthz/ready": dial tcp 10.134.0.86:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Unhealthy

Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

ProbeError

Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-554dd8d747-pfv8n

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-cbf5c5747-867n5

Killing

Stopping container payload-processing
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-b354e3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ipp-b-b354e3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ipp-b-b354e3

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ipp-b-b354e3-645fd85787 from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

Unhealthy

Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-ipp-b-b354e3

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ipp-b-b354e3-558bc9799b from 0 to 1
(x6)

openshift-ingress

kubelet

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

ProbeError

Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused body:
(x9)

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-b354e3-558bc9799b

FailedCreate

Error creating: pods "payload-processing-e2e-ipp-b-b354e3-558bc9799b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b354e3: serviceaccount "payload-processing-e2e-ipp-b-b354e3" not found
(x8)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-b354e3: serviceaccount "payload-processing-e2e-ipp-b-b354e3" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-ipp-b-b354e3-558bc9799b

SuccessfulCreate

Created pod: payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

AddedInterface

Add eth0 [10.133.0.95/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

Pulled

Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

Started

Container started

openshift-ingress

multus

payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

AddedInterface

Add eth0 [10.133.0.96/23] from ovn-kubernetes
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-e7b2a3-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-a-e7b2a3-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-praxis-b-390a75

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-praxis-b-390a75

openshift-ingress

replicaset-controller

e2e-praxis-b-390a75-openshift-default-856d5799f7

SuccessfulCreate

Created pod: e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

openshift-ingress

multus

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

AddedInterface

Add eth0 [10.132.0.77/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-praxis-b-390a75-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-praxis-b-390a75-openshift-default-856d5799f7 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-praxis-b-390a75

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-praxis-b-390a75

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-praxis-b-390a75-78f89c5775 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-390a75-78f89c5775

FailedCreate

Error creating: pods "payload-processing-e2e-praxis-b-390a75-78f89c5775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-390a75: serviceaccount "payload-processing-e2e-praxis-b-390a75" not found

openshift-ingress

multus

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

AddedInterface

Add eth0 [10.134.0.97/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-processing-e2e-praxis-b-390a75-78f89c5775

SuccessfulCreate

Created pod: payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-praxis-b-390a75: serviceaccount "payload-processing-e2e-praxis-b-390a75" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk

AddedInterface

Add eth0 [10.134.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk

Created

Container created
(x2)

llm

kubelet

premium-simulated-simulated-premium-kserve-5f5c4b459f-rf6nx

Unhealthy

Readiness probe failed: Get "https://10.133.0.38:8000/ready": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

ProbeError

Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Unhealthy

Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-390a75-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-390a75-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Unhealthy

Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Unhealthy

Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

ProbeError

Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Unhealthy

Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Unhealthy

Readiness probe failed: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

ProbeError

Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

ProbeError

Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

ProbeError

Readiness probe error: Get "http://10.132.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-25719d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-25719d

openshift-ingress

replicaset-controller

e2e-extmod-a-25719d-openshift-default-77fc995d65

SuccessfulCreate

Created pod: e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

openshift-ingress

deployment-controller

e2e-extmod-a-25719d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-25719d-openshift-default-77fc995d65 from 0 to 1

openshift-ingress

multus

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

AddedInterface

Add eth0 [10.132.0.78/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-25719d-78689cc99f

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-25719d-78689cc99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-25719d: serviceaccount "payload-processing-e2e-extmod-a-25719d" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-25719d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-25719d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-25719d-78689cc99f from 0 to 1

openshift-ingress

multus

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

AddedInterface

Add eth0 [10.134.0.101/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

AddedInterface

Add eth0 [10.134.0.102/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-25719d-78689cc99f

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-25719d: serviceaccount "payload-processing-e2e-extmod-a-25719d" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulDelete

Deleted pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-26j9d

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 3 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-kvv59

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-6zskg

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-kvv59

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-6zskg

openshift-ingress

replicaset-controller

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc

SuccessfulCreate

Created pod: e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

openshift-ingress

deployment-controller

e2e-extmod-b-dc6e48-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-dc6e48-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Started

Container started

openshift-ingress

multus

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

AddedInterface

Add eth0 [10.134.0.104/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Created

Container created

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2
(x5)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

ProbeError

Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x5)

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Unhealthy

Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulCreate

Created pod: istiod-openshift-gateway-7ddd9cfdc8-j7glh

openshift-ingress

multus

istiod-openshift-gateway-7ddd9cfdc8-j7glh

AddedInterface

Add eth0 [10.133.0.109/23] from ovn-kubernetes

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Pulling

Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8"

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Pulled

Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.81s (3.81s including waiting). Image size: 185476477 bytes.

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Started

Container started

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Created

Container created

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-ait-56ae52d5-openshift-default-796558d487

SuccessfulCreate

Created pod: e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

openshift-ingress

gateway_labeler_controller

e2e-ait-56ae52d5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-56ae52d5

openshift-ingress

deployment-controller

e2e-ait-56ae52d5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-56ae52d5-openshift-default-796558d487 from 0 to 1

openshift-ingress

multus

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

AddedInterface

Add eth0 [10.134.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Unhealthy

Readiness probe failed: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

ProbeError

Readiness probe error: Get "http://10.132.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Created

Container created

openshift-ingress

deployment-controller

payload-processing-e2e-ait-56ae52d5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-56ae52d5-d44f5c797 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-56ae52d5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-56ae52d5-d44f5c797

FailedCreate

Error creating: pods "payload-processing-e2e-ait-56ae52d5-d44f5c797-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-56ae52d5: serviceaccount "payload-processing-e2e-ait-56ae52d5" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-56ae52d5-d44f5c797

SuccessfulCreate

Created pod: payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-56ae52d5: serviceaccount "payload-processing-e2e-ait-56ae52d5" not found

openshift-ingress

kubelet

payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

AddedInterface

Add eth0 [10.134.0.109/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2

AddedInterface

Add eth0 [10.134.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

ProbeError

Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Unhealthy

Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-25719d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-25719d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

Unhealthy

Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

ProbeError

Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-a-bd9d24-59cb6c8f87-gxkt9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ipp-a-bd9d24-66d8c6b55b-zb2pl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-ipp-a-bd9d24-openshift-default-6774c4b897-crvgw

Killing

Stopping container istio-proxy
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-390a75-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-praxis-b-390a75-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-25719d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-25719d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-1ff1dc

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-1ff1dc

openshift-ingress

multus

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

AddedInterface

Add eth0 [10.134.0.110/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689

SuccessfulCreate

Created pod: e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

openshift-ingress

deployment-controller

e2e-extmod-b-1ff1dc-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-1ff1dc

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-1ff1dc-7fc785c587 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-1ff1dc

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-1ff1dc: serviceaccount "payload-processing-e2e-extmod-b-1ff1dc" not found
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-1ff1dc: serviceaccount "payload-processing-e2e-extmod-b-1ff1dc" not found

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

AddedInterface

Add eth0 [10.132.0.82/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

Created

Container created

openshift-ingress

multus

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

AddedInterface

Add eth0 [10.134.0.111/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ipp-b-b354e3-558bc9799b-6844x

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ipp-b-b354e3-645fd85787-ndlnq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ipp-b-b354e3-openshift-default-fdb8d8f45-ntsn9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Unhealthy

Readiness probe failed: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

ProbeError

Readiness probe error: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-x4lgh

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-x4lgh
(x5)

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

ProbeError

Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Unhealthy

Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-56ae52d5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-56ae52d5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-476fj

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

AddedInterface

Add eth0 [10.133.0.111/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-c69zs

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-p4lr6

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-c69zs

AddedInterface

Add eth0 [10.134.0.112/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-c69zs

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-476fj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-476fj

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-476fj

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-c69zs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-476fj

AddedInterface

Add eth0 [10.132.0.84/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-c69zs

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

ProbeError

Startup probe error: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-476fj

ProbeError

Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Unhealthy

Startup probe failed: Get "http://10.133.0.111:15021/healthz/ready": dial tcp 10.133.0.111:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-c69zs

Unhealthy

Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-476fj

Unhealthy

Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-c69zs

ProbeError

Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-56ae52d5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-56ae52d5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-56ae52d5-5cf56bddf5-shhdr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-56ae52d5-d44f5c797-mgbm2

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

Unhealthy

Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

ProbeError

Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

Killing

Stopping container payload-pre-processing
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1ff1dc-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1ff1dc-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-n2vdj

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-r9phn

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-fxl5k

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-2tj59

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

AddedInterface

Add eth0 [10.133.0.116/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

AddedInterface

Add eth0 [10.133.0.115/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2tj59

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-r9phn

AddedInterface

Add eth0 [10.132.0.85/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2tj59

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2tj59

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-2tj59

AddedInterface

Add eth0 [10.134.0.113/23] from ovn-kubernetes
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Unhealthy

Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-56ae52d5-openshift-default-796558d487-6gp99

Killing

Stopping container istio-proxy
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

ProbeError

Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2tj59

ProbeError

Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

ProbeError

Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2tj59

Unhealthy

Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

ProbeError

Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

Unhealthy

Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

Unhealthy

Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-migrate-df0a0131-gw

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-df0a0131-gw

openshift-ingress

deployment-controller

e2e-migrate-df0a0131-gw-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f from 0 to 1

openshift-ingress

replicaset-controller

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f

SuccessfulCreate

Created pod: e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

openshift-ingress

multus

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

AddedInterface

Add eth0 [10.132.0.86/23] from ovn-kubernetes
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1ff1dc-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-1ff1dc-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-a-e7b2a3-66d7b4cc6c-m7r48

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-praxis-a-e7b2a3-ffb99b799-lcpkk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-df0a0131-656d84bdb9

FailedCreate

Error creating: pods "payload-processing-e2e-migrate-df0a0131-656d84bdb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-df0a0131: serviceaccount "payload-processing-e2e-migrate-df0a0131" not found

openshift-ingress

deployment-controller

payload-processing-e2e-migrate-df0a0131

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-migrate-df0a0131-656d84bdb9 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-df0a0131: serviceaccount "payload-processing-e2e-migrate-df0a0131" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-migrate-df0a0131

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-migrate-df0a0131-77db945fc9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-migrate-df0a0131-656d84bdb9

SuccessfulCreate

Created pod: payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

Started

Container started

openshift-ingress

multus

payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

AddedInterface

Add eth0 [10.132.0.89/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

AddedInterface

Add eth0 [10.134.0.114/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

openshift-ingress

kubelet

payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-df0a0131-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-df0a0131-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-praxis-a-e7b2a3-openshift-default-578969456f-96nt8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-praxis-b-390a75-78f89c5775-l5vtk

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-praxis-b-390a75-58b567fd8c-pjmpc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w3-2e0116

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-2e0116

openshift-ingress

deployment-controller

e2e-shared-a-w3-2e0116-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w3-2e0116-openshift-default-65b758dfff from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff

SuccessfulCreate

Created pod: e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

multus

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

AddedInterface

Add eth0 [10.133.0.117/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-praxis-b-390a75-openshift-default-856d5799f7-j9ztz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-2ggdr

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-qqlv2

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 10 to 7

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-5sl49

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 7; reason: All metrics below target
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-2e0116: serviceaccount "payload-processing-e2e-shared-a-w3-2e0116" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-2e0116: serviceaccount "payload-processing-e2e-shared-a-w3-2e0116" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w3-2e0116

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w3-2e0116

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

AddedInterface

Add eth0 [10.132.0.91/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

openshift-ingress

multus

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

AddedInterface

Add eth0 [10.132.0.92/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-migrate-df0a0131-656d84bdb9-4kjjt

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-migrate-df0a0131-77db945fc9-txnsx

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Unhealthy

Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Created

Container created

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

ProbeError

Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

ProbeError

Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Unhealthy

Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-df0a0131-gw-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-migrate-df0a0131-gw-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ait-8b4338ad-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-8b4338ad-openshift-default-57f75f84b7 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-8b4338ad-openshift-default-57f75f84b7

SuccessfulCreate

Created pod: e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

openshift-ingress

gateway_labeler_controller

e2e-ait-8b4338ad

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8b4338ad

openshift-ingress

multus

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

AddedInterface

Add eth0 [10.132.0.96/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-migrate-df0a0131-gw-openshift-default-56c44c6f6f-zd88w

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8b4338ad-669b6c55cc

FailedCreate

Error creating: pods "payload-processing-e2e-ait-8b4338ad-669b6c55cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8b4338ad: serviceaccount "payload-processing-e2e-ait-8b4338ad" not found

openshift-ingress

deployment-controller

payload-processing-e2e-ait-8b4338ad

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-8b4338ad-669b6c55cc from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-8b4338ad

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-8b4338ad-659d674775 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-8b4338ad-669b6c55cc

SuccessfulCreate

Created pod: payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

openshift-ingress

multus

payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

AddedInterface

Add eth0 [10.132.0.97/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8b4338ad-659d674775

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

Created

Container created
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-8b4338ad-659d674775

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-8b4338ad-659d674775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8b4338ad: serviceaccount "payload-processing-e2e-ait-8b4338ad" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm

AddedInterface

Add eth0 [10.132.0.98/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

ProbeError

Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

Unhealthy

Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-2e0116-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-2e0116-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-4gqwm

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-r8jnh

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 10 to 7

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-vfwmd

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 7; reason: All metrics below target

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-2e0116-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w3-2e0116-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w3-51aa8f

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-51aa8f

openshift-ingress

deployment-controller

e2e-shared-b-w3-51aa8f-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f from 0 to 1

openshift-ingress

multus

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

AddedInterface

Add eth0 [10.133.0.119/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f

SuccessfulCreate

Created pod: e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-51aa8f: serviceaccount "payload-processing-e2e-shared-b-w3-51aa8f" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w3-51aa8f

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-51aa8f: serviceaccount "payload-processing-e2e-shared-b-w3-51aa8f" not found

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w3-51aa8f

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

AddedInterface

Add eth0 [10.132.0.100/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

AddedInterface

Add eth0 [10.132.0.101/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Created

Container created

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Unhealthy

Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": read tcp 10.132.0.2:46838->10.132.0.96:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

ProbeError

Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": read tcp 10.132.0.2:46838->10.132.0.96:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

ProbeError

Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Unhealthy

Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8b4338ad-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-8b4338ad-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-8b4338ad-659d674775-jvhnm

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-8b4338ad-669b6c55cc-xjtz5

Killing

Stopping container payload-processing
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Unhealthy

Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

ProbeError

Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-51aa8f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-51aa8f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-51aa8f-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w3-51aa8f-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-ait-4c66acdd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-4c66acdd-openshift-default-974d75d7 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-ait-4c66acdd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-4c66acdd

openshift-ingress

replicaset-controller

e2e-ait-4c66acdd-openshift-default-974d75d7

SuccessfulCreate

Created pod: e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

openshift-ingress

multus

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

AddedInterface

Add eth0 [10.132.0.103/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-ait-8b4338ad-openshift-default-57f75f84b7-9nhzz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 10 to 8

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-fxl5k

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-n2vdj

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-n2vdj

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-fxl5k

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-ait-4c66acdd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-4c66acdd-7889576f8f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-4c66acdd-7889576f8f

SuccessfulCreate

Created pod: payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4c66acdd: serviceaccount "payload-processing-e2e-ait-4c66acdd" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-4c66acdd-7889576f8f

FailedCreate

Error creating: pods "payload-processing-e2e-ait-4c66acdd-7889576f8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-4c66acdd: serviceaccount "payload-processing-e2e-ait-4c66acdd" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-4c66acdd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-4c66acdd-9bd86b959 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

openshift-ingress

kubelet

payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

AddedInterface

Add eth0 [10.132.0.105/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf

AddedInterface

Add eth0 [10.132.0.106/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

Started

Container started
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 8 to 6

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-c69zs

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-476fj

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulDelete

Deleted pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 7 to 5

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 5; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 2 to 1

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 7 to 4

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zktj5

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-6dt97

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ll76c

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-lfwxg

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-j958f

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

Unhealthy

Readiness probe failed: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2bx8l

ProbeError

Readiness probe error: Get "http://10.132.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2tj59

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

(combined from similar events): Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-2tj59

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-extmod-b-dc6e48-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc from 2 to 1

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-dc6e48-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc

SuccessfulDelete

Deleted pod: e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-d9mnb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

deployment-controller

istiod-openshift-gateway

ScalingReplicaSet

Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1

openshift-ingress

replicaset-controller

istiod-openshift-gateway-7ddd9cfdc8

SuccessfulDelete

Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-j7glh

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 5 to 4

openshift-ingress

horizontal-pod-autoscaler

istiod-openshift-gateway

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

istiod-openshift-gateway-7ddd9cfdc8-j7glh

Killing

Stopping container discovery

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 4 to 3

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-cffsz

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-9kc55

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x4)

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Unhealthy

Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

ProbeError

Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused body:
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4c66acdd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4c66acdd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4c66acdd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-4c66acdd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulCreate

Created pod: maas-default-gateway-openshift-default-7d9897b54c-njl8w

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

AddedInterface

Add eth0 [10.133.0.128/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-njl8w

AddedInterface

Add eth0 [10.134.0.121/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-4c66acdd-9bd86b959-9gzkh

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-ait-4c66acdd-7889576f8f-x6tqf

Killing

Stopping container payload-processing
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Unhealthy

Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

ProbeError

Startup probe error: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Killing

Stopping container istio-proxy
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

ProbeError

Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 4 to 3

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Unhealthy

Startup probe failed: Get "http://10.134.0.121:15021/healthz/ready": dial tcp 10.134.0.121:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-2pjf2

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

(combined from similar events): New size: 9; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

AddedInterface

Add eth0 [10.132.0.107/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

AddedInterface

Add eth0 [10.134.0.122/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Unhealthy

Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

ProbeError

Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

ProbeError

Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

Unhealthy

Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused
(x5)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 9 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

AddedInterface

Add eth0 [10.133.0.129/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Started

Container started
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

ProbeError

Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Unhealthy

Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-derive-030936cd

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-derive-030936cd

openshift-ingress

multus

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

AddedInterface

Add eth0 [10.133.0.130/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-derive-030936cd-openshift-default-55f6b8f4fc

SuccessfulCreate

Created pod: e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

openshift-ingress

deployment-controller

e2e-derive-030936cd-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-derive-030936cd-openshift-default-55f6b8f4fc from 0 to 1

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-ait-4c66acdd-openshift-default-974d75d7-p2dfs

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-derive-030936cd

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-derive-030936cd-7f9567b94c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-derive-030936cd

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-derive-030936cd-5878568bf9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-030936cd-5878568bf9

SuccessfulCreate

Created pod: payload-processing-e2e-derive-030936cd-5878568bf9-zblpr
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-derive-030936cd-5878568bf9

FailedCreate

Error creating: pods "payload-processing-e2e-derive-030936cd-5878568bf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-030936cd: serviceaccount "payload-processing-e2e-derive-030936cd" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-030936cd-7f9567b94c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-derive-030936cd-7f9567b94c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-derive-030936cd-7f9567b94c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-030936cd: serviceaccount "payload-processing-e2e-derive-030936cd" not found

openshift-ingress

multus

payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2

AddedInterface

Add eth0 [10.132.0.109/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-derive-030936cd-5878568bf9-zblpr

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-derive-030936cd-5878568bf9-zblpr

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-derive-030936cd-5878568bf9-zblpr

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-derive-030936cd-5878568bf9-zblpr

AddedInterface

Add eth0 [10.132.0.108/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2

Started

Container started

opendatahub

multus

test-curl-7695-0ff146

AddedInterface

Add eth0 [10.134.0.123/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7695-0ff146

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

kubelet

test-curl-7695-0ff146

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 1.115s (1.115s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7695-0ff146

Created

Container created

opendatahub

kubelet

test-curl-7695-0ff146

Started

Container started

opendatahub

kubelet

test-curl-7695-0ff146

Killing

Stopping container test-curl-7695-0ff146
(x2)

openshift-ingress

kubelet

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-030936cd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-030936cd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

ProbeError

Startup probe error: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

Unhealthy

Startup probe failed: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused

openshift-ingress

gateway_labeler_controller

e2e-api-a-8a9760

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-8a9760

openshift-ingress

deployment-controller

e2e-api-a-8a9760-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-a-8a9760-openshift-default-57f6f4564c from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-a-8a9760-openshift-default-57f6f4564c

SuccessfulCreate

Created pod: e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

openshift-ingress

multus

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

AddedInterface

Add eth0 [10.133.0.134/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-a-8a9760

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-a-8a9760-588ff9ff9 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-8a9760-998649d7c

SuccessfulCreate

Created pod: payload-processing-e2e-api-a-8a9760-998649d7c-lz57c
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-a-8a9760-998649d7c

FailedCreate

Error creating: pods "payload-processing-e2e-api-a-8a9760-998649d7c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-8a9760: serviceaccount "payload-processing-e2e-api-a-8a9760" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-8a9760: serviceaccount "payload-processing-e2e-api-a-8a9760" not found

openshift-ingress

deployment-controller

payload-processing-e2e-api-a-8a9760

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-a-8a9760-998649d7c from 0 to 1

openshift-ingress

kubelet

payload-processing-e2e-api-a-8a9760-998649d7c-lz57c

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-api-a-8a9760-998649d7c-lz57c

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-api-a-8a9760-998649d7c-lz57c

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

Started

Container started

openshift-ingress

multus

payload-processing-e2e-api-a-8a9760-998649d7c-lz57c

AddedInterface

Add eth0 [10.132.0.111/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

AddedInterface

Add eth0 [10.132.0.110/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-030936cd-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-derive-030936cd-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

openshift-ingress

gateway_labeler_controller

e2e-mt-47bc8f48

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-47bc8f48

openshift-ingress

kubelet

payload-pre-processing-e2e-derive-030936cd-7f9567b94c-cnzt2

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-derive-030936cd-5878568bf9-zblpr

Killing

Stopping container payload-processing

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 2; reason: All metrics below target

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-5lxdx

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 3 to 2

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

(combined from similar events): Scaled down replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 3 to 1

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-bxfjp

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulDelete

Deleted pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-mt-47bc8f48-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-47bc8f48-openshift-default-7767fd945c from 0 to 1

openshift-ingress

replicaset-controller

e2e-mt-47bc8f48-openshift-default-7767fd945c

SuccessfulCreate

Created pod: e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

openshift-ingress

multus

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

AddedInterface

Add eth0 [10.133.0.143/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-k7sxq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-47bc8f48

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-47bc8f48-789f69b859 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-47bc8f48

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-47bc8f48-65dd45f78c from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-47bc8f48-65dd45f78c

FailedCreate

Error creating: pods "payload-processing-e2e-mt-47bc8f48-65dd45f78c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-47bc8f48: serviceaccount "payload-processing-e2e-mt-47bc8f48" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-47bc8f48-789f69b859

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-47bc8f48-789f69b859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-47bc8f48: serviceaccount "payload-processing-e2e-mt-47bc8f48" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-47bc8f48-65dd45f78c

SuccessfulCreate

Created pod: payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

openshift-ingress

multus

payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

AddedInterface

Add eth0 [10.132.0.112/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-47bc8f48-789f69b859

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

openshift-ingress

multus

payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

AddedInterface

Add eth0 [10.132.0.113/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

Created

Container created

openshift-ingress

kubelet

e2e-derive-030936cd-openshift-default-55f6b8f4fc-mzn89

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669

SuccessfulCreate

Created pod: e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

openshift-ingress

deployment-controller

e2e-extmod-a-caf74d-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-a-caf74d-openshift-default-6bd6bf9669 from 0 to 1

openshift-ingress

gateway_labeler_controller

e2e-extmod-a-caf74d

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-a-caf74d

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

ProbeError

Readiness probe error: Get "http://10.133.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Unhealthy

Readiness probe failed: Get "http://10.133.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

AddedInterface

Add eth0 [10.133.0.144/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

ProbeError

Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Unhealthy

Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-caf74d: serviceaccount "payload-processing-e2e-extmod-a-caf74d" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-a-caf74d

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-a-caf74d: serviceaccount "payload-processing-e2e-extmod-a-caf74d" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-a-caf74d

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-a-caf74d-55b6d48fb9 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

openshift-ingress

multus

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

AddedInterface

Add eth0 [10.132.0.115/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

AddedInterface

Add eth0 [10.132.0.116/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-8a9760-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-8a9760-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-8a9760-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-a-8a9760-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

gateway_labeler_controller

e2e-api-b-d855db

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-d855db

openshift-ingress

deployment-controller

e2e-api-b-d855db-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-api-b-d855db-openshift-default-577cbd8699 from 0 to 1

openshift-ingress

replicaset-controller

e2e-api-b-d855db-openshift-default-577cbd8699

SuccessfulCreate

Created pod: e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

openshift-ingress

multus

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

AddedInterface

Add eth0 [10.133.0.145/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

ProbeError

Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

Unhealthy

Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused

openshift-ingress

deployment-controller

payload-processing-e2e-api-b-d855db

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-api-b-d855db-76f476689 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-d855db-76f476689

SuccessfulCreate

Created pod: payload-processing-e2e-api-b-d855db-76f476689-zjv2j
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-api-b-d855db-76f476689

FailedCreate

Error creating: pods "payload-processing-e2e-api-b-d855db-76f476689-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d855db: serviceaccount "payload-processing-e2e-api-b-d855db" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-api-b-d855db

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-api-b-d855db-658cb4ffd7 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7

SuccessfulCreate

Created pod: payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7

FailedCreate

Error creating: pods "payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-d855db: serviceaccount "payload-processing-e2e-api-b-d855db" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-api-b-d855db-76f476689-zjv2j

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-api-b-d855db-76f476689-zjv2j

AddedInterface

Add eth0 [10.134.0.130/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-api-b-d855db-76f476689-zjv2j

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-api-b-d855db-76f476689-zjv2j

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx

AddedInterface

Add eth0 [10.132.0.118/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx

Started

Container started
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-47bc8f48-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-47bc8f48-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

Created

Container created
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-caf74d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-caf74d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

Unhealthy

Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

ProbeError

Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-caf74d-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-a-caf74d-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

deployment-controller

e2e-extmod-b-581487-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-extmod-b-581487-openshift-default-755f5b8d9c from 0 to 1

openshift-ingress

replicaset-controller

e2e-extmod-b-581487-openshift-default-755f5b8d9c

SuccessfulCreate

Created pod: e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

openshift-ingress

gateway_labeler_controller

e2e-extmod-b-581487

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-extmod-b-581487

openshift-ingress

multus

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

AddedInterface

Add eth0 [10.133.0.158/23] from ovn-kubernetes

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7446cdb47f from 0 to 1
(x7)

models-as-a-service

replicaset-controller

payload-processing-external-model-7446cdb47f

FailedCreate

Error creating: pods "payload-processing-external-model-7446cdb47f-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

replicaset-controller

payload-processing-external-model-7446cdb47f

SuccessfulCreate

Created pod: payload-processing-external-model-7446cdb47f-xqntm

models-as-a-service

multus

payload-processing-external-model-7446cdb47f-xqntm

AddedInterface

Add eth0 [10.132.0.126/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7446cdb47f-xqntm

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-7446cdb47f-xqntm

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-7446cdb47f-xqntm

Started

Container started

openshift-ingress

deployment-controller

payload-pre-processing-e2e-extmod-b-581487

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-extmod-b-581487

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-extmod-b-581487-f88795567 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c

SuccessfulCreate

Created pod: payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c

FailedCreate

Error creating: pods "payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-581487: serviceaccount "payload-processing-e2e-extmod-b-581487" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-581487-f88795567

SuccessfulCreate

Created pod: payload-processing-e2e-extmod-b-581487-f88795567-8kgw2
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-extmod-b-581487-f88795567

FailedCreate

Error creating: pods "payload-processing-e2e-extmod-b-581487-f88795567-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-extmod-b-581487: serviceaccount "payload-processing-e2e-extmod-b-581487" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

AddedInterface

Add eth0 [10.132.0.127/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt

AddedInterface

Add eth0 [10.134.0.134/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

Created

Container created

models-as-a-service

replicaset-controller

payload-processing-external-model-595cb556d4

SuccessfulCreate

Created pod: payload-processing-external-model-595cb556d4-tfnm9

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-595cb556d4 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-595cb556d4-tfnm9

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-595cb556d4-tfnm9

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-595cb556d4-tfnm9

Created

Container created

models-as-a-service

multus

payload-processing-external-model-595cb556d4-tfnm9

AddedInterface

Add eth0 [10.132.0.128/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-7446cdb47f

SuccessfulDelete

Deleted pod: payload-processing-external-model-7446cdb47f-xqntm

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-7446cdb47f from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-7446cdb47f-xqntm

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-787d5c596d

SuccessfulCreate

Created pod: payload-processing-external-model-787d5c596d-gsfjj

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-787d5c596d from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-787d5c596d-gsfjj

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-787d5c596d-gsfjj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-787d5c596d-gsfjj

Started

Container started

models-as-a-service

multus

payload-processing-external-model-787d5c596d-gsfjj

AddedInterface

Add eth0 [10.132.0.129/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-47bc8f48-789f69b859-sk2rt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-47bc8f48-65dd45f78c-8n476

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-d855db-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-d855db-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-pre-processing-e2e-api-a-8a9760-588ff9ff9-8cssv

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-api-a-8a9760-998649d7c-lz57c

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Unhealthy

Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

ProbeError

Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused body:
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-d855db-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-api-b-d855db-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-mt-47bc8f48-openshift-default-7767fd945c-q2cj6

Killing

Stopping container istio-proxy

models-as-a-service

replicaset-controller

payload-processing-external-model-787d5c596d

SuccessfulDelete

Deleted pod: payload-processing-external-model-787d5c596d-gsfjj

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-7b968cbf54 from 0 to 1
(x5)

models-as-a-service

kubelet

payload-processing-external-model-787d5c596d-gsfjj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

replicaset-controller

payload-processing-external-model-7b968cbf54

SuccessfulCreate

Created pod: payload-processing-external-model-7b968cbf54-hrgz2

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-787d5c596d from 1 to 0

models-as-a-service

kubelet

payload-processing-external-model-787d5c596d-gsfjj

Killing

Stopping container payload-processing

models-as-a-service

kubelet

payload-processing-external-model-7b968cbf54-hrgz2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-7b968cbf54-hrgz2

AddedInterface

Add eth0 [10.132.0.131/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-7b968cbf54-hrgz2

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-7b968cbf54-hrgz2

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

Started

Container started

openshift-ingress

kubelet

e2e-api-a-8a9760-openshift-default-57f6f4564c-pcwsc

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

ProbeError

Startup probe error: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

Unhealthy

Startup probe failed: Get "http://10.133.0.158:15021/healthz/ready": dial tcp 10.133.0.158:15021: connect: connection refused
(x2)

models-as-a-service

kubelet

payload-processing-external-model-7b968cbf54-hrgz2

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-7b968cbf54-hrgz2

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-7b968cbf54

SuccessfulDelete

Deleted pod: payload-processing-external-model-7b968cbf54-hrgz2

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-7b968cbf54 from 1 to 0

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-64954bd78b from 0 to 1

models-as-a-service

replicaset-controller

payload-processing-external-model-64954bd78b

SuccessfulCreate

Created pod: payload-processing-external-model-64954bd78b-zr94k

models-as-a-service

kubelet

payload-processing-external-model-64954bd78b-zr94k

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-64954bd78b-zr94k

Started

Container started

models-as-a-service

multus

payload-processing-external-model-64954bd78b-zr94k

AddedInterface

Add eth0 [10.132.0.132/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-64954bd78b-zr94k

Created

Container created

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w0-302ba0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-302ba0

openshift-ingress

deployment-controller

e2e-shared-a-w0-302ba0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff

SuccessfulCreate

Created pod: e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

openshift-ingress

kubelet

payload-processing-e2e-api-b-d855db-76f476689-zjv2j

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-api-b-d855db-658cb4ffd7-rwskx

Killing

Stopping container payload-pre-processing

openshift-ingress

multus

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

AddedInterface

Add eth0 [10.133.0.159/23] from ovn-kubernetes
(x6)

models-as-a-service

kubelet

payload-processing-external-model-595cb556d4-tfnm9

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-95e911575933" not found

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Unhealthy

Readiness probe failed: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

ProbeError

Readiness probe error: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w0-302ba0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w0-302ba0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-302ba0: serviceaccount "payload-processing-e2e-shared-a-w0-302ba0" not found

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

AddedInterface

Add eth0 [10.132.0.133/23] from ovn-kubernetes

models-as-a-service

replicaset-controller

payload-processing-external-model-595cb556d4

SuccessfulDelete

Deleted pod: payload-processing-external-model-595cb556d4-tfnm9
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-302ba0: serviceaccount "payload-processing-e2e-shared-a-w0-302ba0" not found

models-as-a-service

kubelet

payload-processing-external-model-595cb556d4-tfnm9

Killing

Stopping container payload-processing

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-595cb556d4 from 1 to 0

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

AddedInterface

Add eth0 [10.132.0.134/23] from ovn-kubernetes
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-581487-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-extmod-b-581487-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulCreate

Created pod: payload-processing-external-model-866d5b9c86-vpmlw

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-vpmlw

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-vpmlw

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-vpmlw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-866d5b9c86-vpmlw

AddedInterface

Add eth0 [10.132.0.135/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-vpmlw

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-985777bbf

SuccessfulCreate

Created pod: payload-processing-external-model-985777bbf-ln7cv

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulDelete

Deleted pod: payload-processing-external-model-866d5b9c86-vpmlw

models-as-a-service

kubelet

payload-processing-external-model-985777bbf-ln7cv

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-985777bbf-ln7cv

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-985777bbf-ln7cv

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-985777bbf-ln7cv

AddedInterface

Add eth0 [10.132.0.136/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Unhealthy

Readiness probe failed: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

ProbeError

Readiness probe error: Get "http://10.133.0.111:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

models-as-a-service

kubelet

payload-processing-external-model-985777bbf-ln7cv

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-985777bbf

SuccessfulDelete

Deleted pod: payload-processing-external-model-985777bbf-ln7cv

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulCreate

Created pod: payload-processing-external-model-866d5b9c86-qm2hn
(x4)

models-as-a-service

kubelet

payload-processing-external-model-985777bbf-ln7cv

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-75fbcfa535aa" not found

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-qm2hn

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-qm2hn

Started

Container started

models-as-a-service

multus

payload-processing-external-model-866d5b9c86-qm2hn

AddedInterface

Add eth0 [10.132.0.137/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-qm2hn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulDelete

Deleted pod: payload-processing-external-model-866d5b9c86-qm2hn

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-qm2hn

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

replicaset-controller

payload-processing-external-model-985777bbf

SuccessfulCreate

Created pod: payload-processing-external-model-985777bbf-wxk2d

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-qm2hn

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulCreate

Created pod: payload-processing-external-model-866d5b9c86-lp22f

models-as-a-service

replicaset-controller

payload-processing-external-model-985777bbf

SuccessfulDelete

Deleted pod: payload-processing-external-model-985777bbf-wxk2d
(x4)

models-as-a-service

kubelet

payload-processing-external-model-985777bbf-wxk2d

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-75fbcfa535aa" not found

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-lp22f

Started

Container started

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Killing

Stopping container istio-proxy

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-lp22f

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-lp22f

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

multus

payload-processing-external-model-866d5b9c86-lp22f

AddedInterface

Add eth0 [10.134.0.136/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-worker-w3-28c598

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-28c598

openshift-ingress

deployment-controller

e2e-worker-w3-28c598-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-28c598-openshift-default-7f67789789 from 0 to 1

openshift-ingress

replicaset-controller

e2e-worker-w3-28c598-openshift-default-7f67789789

SuccessfulCreate

Created pod: e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

openshift-ingress

multus

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

AddedInterface

Add eth0 [10.133.0.161/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-api-b-d855db-openshift-default-577cbd8699-6djwm

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

models-as-a-service

multus

payload-processing-external-model-6bd55dd7d7-7swd4

AddedInterface

Add eth0 [10.134.0.137/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-6bd55dd7d7-7swd4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulDelete

Deleted pod: payload-processing-external-model-866d5b9c86-lp22f

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-lp22f

Killing

Stopping container payload-processing

models-as-a-service

replicaset-controller

payload-processing-external-model-6bd55dd7d7

SuccessfulCreate

Created pod: payload-processing-external-model-6bd55dd7d7-7swd4

models-as-a-service

kubelet

payload-processing-external-model-6bd55dd7d7-7swd4

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-6bd55dd7d7-7swd4

Created

Container created

models-as-a-service

replicaset-controller

payload-processing-external-model-6bd55dd7d7

SuccessfulDelete

Deleted pod: payload-processing-external-model-6bd55dd7d7-7swd4

openshift-ingress

gateway_labeler_controller

e2e-autores-a3e841

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-autores-a3e841

models-as-a-service

kubelet

payload-processing-external-model-6bd55dd7d7-7swd4

Killing

Stopping container payload-processing
(x2)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-866d5b9c86 from 0 to 1

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-dxh9q

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

e2e-autores-a3e841-openshift-default-6989c98fc5

SuccessfulCreate

Created pod: e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulCreate

Created pod: payload-processing-external-model-866d5b9c86-dxh9q

openshift-ingress

deployment-controller

e2e-autores-a3e841-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-autores-a3e841-openshift-default-6989c98fc5 from 0 to 1

models-as-a-service

multus

payload-processing-external-model-866d5b9c86-dxh9q

AddedInterface

Add eth0 [10.134.0.138/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-dxh9q

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-dxh9q

Created

Container created

openshift-ingress

multus

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

AddedInterface

Add eth0 [10.133.0.162/23] from ovn-kubernetes
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-28c598: serviceaccount "payload-processing-e2e-worker-w3-28c598" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-28c598-55548bdb55

FailedCreate

Error creating: pods "payload-processing-e2e-worker-w3-28c598-55548bdb55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-28c598: serviceaccount "payload-processing-e2e-worker-w3-28c598" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-w3-28c598

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-w3-28c598-69f558c87f from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-worker-w3-28c598

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-w3-28c598-55548bdb55 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-w3-28c598-55548bdb55

SuccessfulCreate

Created pod: payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

Created

Container created

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

AddedInterface

Add eth0 [10.134.0.139/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

AddedInterface

Add eth0 [10.134.0.140/23] from ovn-kubernetes
(x5)

models-as-a-service

kubelet

payload-processing-external-model-6bd55dd7d7-7swd4

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-0121f247d8fb" not found

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

Started

Container started

openshift-ingress

deployment-controller

payload-processing-e2e-autores-a3e841

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-autores-a3e841-5d4f79784f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-autores-a3e841

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-autores-a3e841-6d597c6575 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-a3e841-6d597c6575

SuccessfulCreate

Created pod: payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-a3e841-5d4f79784f

FailedCreate

Error creating: pods "payload-processing-e2e-autores-a3e841-5d4f79784f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-a3e841: serviceaccount "payload-processing-e2e-autores-a3e841" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-autores-a3e841-6d597c6575

FailedCreate

Error creating: pods "payload-pre-processing-e2e-autores-a3e841-6d597c6575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-autores-a3e841: serviceaccount "payload-processing-e2e-autores-a3e841" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-autores-a3e841-5d4f79784f

SuccessfulCreate

Created pod: payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

openshift-ingress

multus

payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

AddedInterface

Add eth0 [10.132.0.140/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng

AddedInterface

Add eth0 [10.134.0.141/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng

Started

Container started

openshift-ingress

deployment-controller

e2e-shared-a-w0-302ba0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff from 1 to 2

openshift-ingress

replicaset-controller

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff

SuccessfulCreate

Created pod: e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-302ba0-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

Created

Container created

openshift-ingress

multus

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

AddedInterface

Add eth0 [10.132.0.141/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Unhealthy

Readiness probe failed: Get "http://10.133.0.159:15021/healthz/ready": read tcp 10.133.0.2:47522->10.133.0.159:15021: read: connection reset by peer

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

ProbeError

Readiness probe error: Get "http://10.133.0.159:15021/healthz/ready": read tcp 10.133.0.2:47522->10.133.0.159:15021: read: connection reset by peer body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

ProbeError

Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

Unhealthy

Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

ProbeError

Startup probe error: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Unhealthy

Startup probe failed: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-302ba0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w0-302ba0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w0-6ba6e5

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-6ba6e5

openshift-ingress

deployment-controller

e2e-shared-b-w0-6ba6e5-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff from 0 to 1

openshift-ingress

replicaset-controller

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff

SuccessfulCreate

Created pod: e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

openshift-ingress

multus

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

AddedInterface

Add eth0 [10.133.0.164/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w0-6ba6e5

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-6ba6e5: serviceaccount "payload-processing-e2e-shared-b-w0-6ba6e5" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-6ba6e5: serviceaccount "payload-processing-e2e-shared-b-w0-6ba6e5" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w0-6ba6e5

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59 from 0 to 1

openshift-ingress

multus

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

AddedInterface

Add eth0 [10.134.0.143/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

AddedInterface

Add eth0 [10.132.0.142/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

ProbeError

Startup probe error: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Unhealthy

Startup probe failed: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

ProbeError

Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Unhealthy

Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-a3e841-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-a3e841-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-a3e841-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-autores-a3e841-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x9)

models-as-a-service

kubelet

payload-processing-external-model-64954bd78b-zr94k

FailedMount

MountVolume.SetUp failed for volume "provider-credentials" : secret "agc-cred-6883279a9766" not found
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-28c598-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x6)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-28c598-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

kubelet

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w0-6ba6e5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w0-6ba6e5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x4)

openshift-ingress

kubelet

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

Unhealthy

Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused
(x4)

openshift-ingress

kubelet

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

ProbeError

Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused body:

models-as-a-service

replicaset-controller

payload-processing-external-model-66c9f9b6bc

SuccessfulCreate

Created pod: payload-processing-external-model-66c9f9b6bc-q27lf
(x3)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled down replica set payload-processing-external-model-866d5b9c86 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-866d5b9c86

SuccessfulDelete

Deleted pod: payload-processing-external-model-866d5b9c86-dxh9q

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-dxh9q

Killing

Stopping container payload-processing
(x20)

models-as-a-service

kubelet

payload-processing-external-model-866d5b9c86-dxh9q

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x10)

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

(combined from similar events): Scaled up replica set payload-processing-external-model-66c9f9b6bc from 0 to 1

models-as-a-service

multus

payload-processing-external-model-66c9f9b6bc-q27lf

AddedInterface

Add eth0 [10.132.0.144/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-66c9f9b6bc-q27lf

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-66c9f9b6bc-q27lf

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-66c9f9b6bc-q27lf

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-66c9f9b6bc-q27lf

Killing

Stopping container payload-processing
(x2)

models-as-a-service

kubelet

payload-processing-external-model-66c9f9b6bc-q27lf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

models-as-a-service

kubelet

payload-processing-external-model-64954bd78b-zr94k

Killing

Stopping container payload-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w0-6ba6e5-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w0-6ba6e5-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kubelet

test-curl-7686-3d97be

Created

Container created

opendatahub

multus

test-curl-7686-3d97be

AddedInterface

Add eth0 [10.134.0.146/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7686-3d97be

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7686-3d97be

Started

Container started

opendatahub

kubelet

test-curl-7686-3d97be

Killing

Stopping container test-curl-7686-3d97be

opendatahub

multus

test-curl-7686-046fbf

AddedInterface

Add eth0 [10.132.0.147/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7686-046fbf

Pulling

Pulling image "registry.access.redhat.com/ubi9/ubi-minimal:9.5"

opendatahub

kubelet

test-curl-7686-046fbf

Pulled

Successfully pulled image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" in 2.877s (2.877s including waiting). Image size: 106100986 bytes.

opendatahub

kubelet

test-curl-7686-046fbf

Created

Container created

opendatahub

kubelet

test-curl-7686-046fbf

Started

Container started

opendatahub

kubelet

test-curl-7686-99155e

Created

Container created

opendatahub

kubelet

test-curl-7686-046fbf

Killing

Stopping container test-curl-7686-046fbf

opendatahub

kubelet

test-curl-7686-99155e

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7686-99155e

AddedInterface

Add eth0 [10.132.0.148/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7686-99155e

Started

Container started

opendatahub

kubelet

test-curl-7686-99155e

Killing

Stopping container test-curl-7686-99155e

opendatahub

kubelet

test-curl-7686-13cf87

Started

Container started

opendatahub

kubelet

test-curl-7686-13cf87

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

multus

test-curl-7686-13cf87

AddedInterface

Add eth0 [10.132.0.149/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7686-13cf87

Created

Container created

opendatahub

kubelet

test-curl-7686-13cf87

Killing

Stopping container test-curl-7686-13cf87

openshift-ingress

replicaset-controller

e2e-worker-w3-28c598-openshift-default-7f67789789

SuccessfulCreate

Created pod: e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

openshift-ingress

deployment-controller

e2e-worker-w3-28c598-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w3-28c598-openshift-default-7f67789789 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-28c598-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

gateway_labeler_controller

e2e-shared-a-w2-3db672

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-3db672

openshift-ingress

replicaset-controller

e2e-shared-a-w2-3db672-openshift-default-c799448c

SuccessfulCreate

Created pod: e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

openshift-ingress

multus

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

AddedInterface

Add eth0 [10.132.0.150/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

Created

Container created

openshift-ingress

deployment-controller

e2e-shared-a-w2-3db672-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-3db672-openshift-default-c799448c from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-autores-a3e841-6d597c6575-nn9ng

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-autores-a3e841-5d4f79784f-pgmlt

Killing

Stopping container payload-processing

openshift-ingress

multus

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

AddedInterface

Add eth0 [10.133.0.171/23] from ovn-kubernetes
(x6)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

ProbeError

Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

Unhealthy

Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-3db672: serviceaccount "payload-processing-e2e-shared-a-w2-3db672" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-a-w2-3db672

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65

FailedCreate

Error creating: pods "payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-3db672: serviceaccount "payload-processing-e2e-shared-a-w2-3db672" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

openshift-ingress

deployment-controller

payload-processing-e2e-shared-a-w2-3db672

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-a-w2-3db672-86f55c8c65 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

Created

Container created

openshift-ingress

multus

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

AddedInterface

Add eth0 [10.134.0.148/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65

SuccessfulCreate

Created pod: payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

AddedInterface

Add eth0 [10.132.0.151/23] from ovn-kubernetes

openshift-ingress

gateway_labeler_controller

e2e-mt-c831a4c0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-mt-c831a4c0

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

openshift-ingress

deployment-controller

e2e-mt-c831a4c0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-c831a4c0-openshift-default-cbd6b4ddf from 0 to 1

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

AddedInterface

Add eth0 [10.133.0.172/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-c831a4c0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-mt-c831a4c0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-c831a4c0-6fdb86cf8f from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c831a4c0: serviceaccount "payload-processing-e2e-mt-c831a4c0" not found

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f

SuccessfulCreate

Created pod: payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

openshift-ingress

multus

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

AddedInterface

Add eth0 [10.132.0.152/23] from ovn-kubernetes
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f

FailedCreate

Error creating: pods "payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c831a4c0: serviceaccount "payload-processing-e2e-mt-c831a4c0" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

Started

Container started

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

AddedInterface

Add eth0 [10.134.0.150/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

Created

Container created

openshift-ingress

deployment-controller

e2e-explicit-0cb4ef-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69 from 0 to 1

openshift-ingress

replicaset-controller

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69

SuccessfulCreate

Created pod: e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

openshift-ingress

gateway_labeler_controller

e2e-explicit-0cb4ef

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-explicit-0cb4ef

openshift-ingress

multus

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

AddedInterface

Add eth0 [10.133.0.173/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-autores-a3e841-openshift-default-6989c98fc5-q5s9v

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-0cb4ef-cc64466f

FailedCreate

Error creating: pods "payload-processing-e2e-explicit-0cb4ef-cc64466f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-0cb4ef: serviceaccount "payload-processing-e2e-explicit-0cb4ef" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9

FailedCreate

Error creating: pods "payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-explicit-0cb4ef: serviceaccount "payload-processing-e2e-explicit-0cb4ef" not found

openshift-ingress

deployment-controller

payload-processing-e2e-explicit-0cb4ef

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-explicit-0cb4ef-cc64466f from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-explicit-0cb4ef

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9 from 0 to 1

openshift-ingress

multus

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

AddedInterface

Add eth0 [10.134.0.151/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

Started

Container started

openshift-ingress

replicaset-controller

payload-processing-e2e-explicit-0cb4ef-cc64466f

SuccessfulCreate

Created pod: payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

openshift-ingress

multus

payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

AddedInterface

Add eth0 [10.132.0.153/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9

SuccessfulCreate

Created pod: payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

openshift-ingress

kubelet

payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

Started

Container started

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w2-3db672-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-shared-a-w2-3db672-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-a-w2-3db672-openshift-default-c799448c from 1 to 2

openshift-ingress

replicaset-controller

e2e-shared-a-w2-3db672-openshift-default-c799448c

SuccessfulCreate

Created pod: e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

openshift-ingress

multus

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

AddedInterface

Add eth0 [10.134.0.153/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Created

Container created

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Started

Container started

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Unhealthy

Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

ProbeError

Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

Started

Container started
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

ProbeError

Startup probe error: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

Unhealthy

Startup probe failed: Get "http://10.133.0.171:15021/healthz/ready": dial tcp 10.133.0.171:15021: connect: connection refused
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w2-3db672-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w2-3db672-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w2-3db672-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w2-3db672-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

ProbeError

Readiness probe error: Get "http://10.133.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Unhealthy

Readiness probe failed: Get "http://10.133.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x5)

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

ProbeError

Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

ProbeError

Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Unhealthy

Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

Unhealthy

Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-0cb4ef-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-explicit-0cb4ef-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

gateway_labeler_controller

e2e-shared-b-w2-c52bfa

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-c52bfa

openshift-ingress

replicaset-controller

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799

SuccessfulCreate

Created pod: e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

openshift-ingress

deployment-controller

e2e-shared-b-w2-c52bfa-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799 from 0 to 1

openshift-ingress

multus

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

AddedInterface

Add eth0 [10.133.0.174/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-pre-processing-e2e-shared-b-w2-c52bfa

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579 from 0 to 1

openshift-ingress

deployment-controller

payload-processing-e2e-shared-b-w2-c52bfa

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc

FailedCreate

Error creating: pods "payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-c52bfa: serviceaccount "payload-processing-e2e-shared-b-w2-c52bfa" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579

SuccessfulCreate

Created pod: payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

openshift-ingress

replicaset-controller

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc

SuccessfulCreate

Created pod: payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579

FailedCreate

Error creating: pods "payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-c52bfa: serviceaccount "payload-processing-e2e-shared-b-w2-c52bfa" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

AddedInterface

Add eth0 [10.134.0.154/23] from ovn-kubernetes

openshift-ingress

multus

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76

AddedInterface

Add eth0 [10.132.0.156/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76

Started

Container started

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-dwr2h

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-g2829

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-g2829

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-p4lr6
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-p4lr6

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 10 to 6

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-7jm7x

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7jm7x

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-dwr2h

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

payload-processing-e2e-explicit-0cb4ef-cc64466f-gphkc

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-explicit-0cb4ef-cd664b6f9-5wvkj

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

Started

Container started

opendatahub

kubelet

test-curl-7695-c60e8b

Created

Container created

opendatahub

kubelet

test-curl-7695-c60e8b

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7695-c60e8b

Started

Container started

opendatahub

multus

test-curl-7695-c60e8b

AddedInterface

Add eth0 [10.133.0.192/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7695-c60e8b

Killing

Stopping container test-curl-7695-c60e8b
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

ProbeError

Startup probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

Unhealthy

Startup probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 2 to 1

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulDelete

Deleted pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-cc8ng

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

Killing

Stopping container payload-pre-processing
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w2-c52bfa-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w2-c52bfa-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

e2e-worker-w3-28c598-openshift-default-7f67789789

SuccessfulDelete

Deleted pod: e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w3-28c598-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

deployment-controller

e2e-worker-w3-28c598-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-worker-w3-28c598-openshift-default-7f67789789 from 2 to 1

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-5n8j7

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w2-c52bfa-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-b-w2-c52bfa-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-qjvzp

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-5wd2n

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 6 to 4

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 4; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-d9bdq

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-njl8w

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w2-3db672-c5cfb8f7f-k8b4x

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w2-3db672-86f55c8c65-whcqn

Killing

Stopping container payload-processing

openshift-ingress

gateway_labeler_controller

e2e-noten-8e1f56

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-noten-8e1f56

openshift-ingress

kubelet

e2e-explicit-0cb4ef-openshift-default-6d6d6dcb69-4hg4w

Killing

Stopping container istio-proxy

openshift-ingress

multus

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

AddedInterface

Add eth0 [10.133.0.196/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-noten-8e1f56-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-noten-8e1f56-openshift-default-64bf794c5f from 0 to 1

openshift-ingress

replicaset-controller

e2e-noten-8e1f56-openshift-default-64bf794c5f

SuccessfulCreate

Created pod: e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

openshift-ingress

kubelet

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

Created

Container created

openshift-ingress

kubelet

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

Started

Container started

opendatahub

kubelet

test-curl-7686-254a3e

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7686-254a3e

Started

Container started

opendatahub

multus

test-curl-7686-254a3e

AddedInterface

Add eth0 [10.133.0.198/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7686-254a3e

Created

Container created

openshift-ingress

deployment-controller

e2e-shared-a-w2-3db672-openshift-default

ScalingReplicaSet

Scaled down replica set e2e-shared-a-w2-3db672-openshift-default-c799448c from 2 to 1

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-shared-a-w2-3db672-openshift-default

SuccessfulRescale

New size: 1; reason: All metrics below target

openshift-ingress

replicaset-controller

e2e-shared-a-w2-3db672-openshift-default-c799448c

SuccessfulDelete

Deleted pod: e2e-shared-a-w2-3db672-openshift-default-c799448c-hsvw5

opendatahub

multus

test-curl-7686-3f6539

AddedInterface

Add eth0 [10.133.0.199/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7686-254a3e

Killing

Stopping container test-curl-7686-254a3e

opendatahub

kubelet

test-curl-7686-3f6539

Started

Container started

opendatahub

kubelet

test-curl-7686-3f6539

Created

Container created

opendatahub

kubelet

test-curl-7686-3f6539

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-8e1f56-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-noten-8e1f56-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

kubelet

test-curl-7686-3f6539

Killing

Stopping container test-curl-7686-3f6539

openshift-ingress

kubelet

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-noten-8e1f56-openshift-default-64bf794c5f-55kxw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-a-w2-3db672-openshift-default-c799448c-ftpr9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w2-c52bfa-78d4465dfc-rgz76

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w2-c52bfa-7f95745579-kvlp7

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w2-c52bfa-openshift-default-78b4d47799-2m4jp

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

payload-processing-e2e-mt-c831a4c0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-mt-c831a4c0-6fdb86cf8f from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f

SuccessfulCreate

Created pod: payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8
(x7)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5

FailedCreate

Error creating: pods "payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c831a4c0: serviceaccount "payload-processing-e2e-mt-c831a4c0" not found

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5

SuccessfulCreate

Created pod: payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

openshift-ingress

deployment-controller

payload-pre-processing-e2e-mt-c831a4c0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5 from 0 to 1
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f

FailedCreate

Error creating: pods "payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-c831a4c0: serviceaccount "payload-processing-e2e-mt-c831a4c0" not found

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

Started

Container started

openshift-ingress

multus

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

AddedInterface

Add eth0 [10.133.0.206/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8

Started

Container started

openshift-ingress

multus

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8

AddedInterface

Add eth0 [10.133.0.205/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-c79abe-5dc676ccd5-kf2pk

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-c79abe-74c56b847f-nknsb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-a-c79abe-openshift-default-94746d989-qknkn

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-dc6e48-6ff9c58855-5q79r

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 4 to 3

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-r9phn

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-r9phn
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-dc6e48-6869d88cd5-lz2sf

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-extmod-b-dc6e48-openshift-default-86d7ccfcc-vvrc5

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 1 to 2

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Created

Container created

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

AddedInterface

Add eth0 [10.133.0.209/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

AddedInterface

Add eth0 [10.133.0.210/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

AddedInterface

Add eth0 [10.134.0.162/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-mt-c831a4c0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-c831a4c0-openshift-default-cbd6b4ddf from 1 to 4

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

Created

Container created

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

AddedInterface

Add eth0 [10.132.0.160/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

Started

Container started

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Unhealthy

Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

ProbeError

Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

Unhealthy

Startup probe failed: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

ProbeError

Startup probe error: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

Unhealthy

Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

ProbeError

Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

ProbeError

Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

Unhealthy

Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

openshift-ingress

deployment-controller

e2e-mt-c831a4c0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-c831a4c0-openshift-default-cbd6b4ddf from 4 to 8

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

AddedInterface

Add eth0 [10.132.0.161/23] from ovn-kubernetes

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

AddedInterface

Add eth0 [10.133.0.212/23] from ovn-kubernetes

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

AddedInterface

Add eth0 [10.132.0.162/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

SuccessfulRescale

New size: 8; reason:

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

AddedInterface

Add eth0 [10.134.0.163/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

ProbeError

Startup probe error: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

ProbeError

Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

Unhealthy

Startup probe failed: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

ProbeError

Startup probe error: Get "http://10.132.0.162:15021/healthz/ready": dial tcp 10.132.0.162:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

Unhealthy

Startup probe failed: Get "http://10.133.0.212:15021/healthz/ready": dial tcp 10.133.0.212:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

ProbeError

Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

Unhealthy

Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

Unhealthy

Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

(combined from similar events): Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

openshift-ingress

horizontal-pod-autoscaler

e2e-mt-c831a4c0-openshift-default

SuccessfulRescale

New size: 10; reason:

openshift-ingress

deployment-controller

e2e-mt-c831a4c0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-mt-c831a4c0-openshift-default-cbd6b4ddf from 8 to 10

openshift-ingress

replicaset-controller

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf

SuccessfulCreate

Created pod: e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

AddedInterface

Add eth0 [10.133.0.213/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

Started

Container started

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

Created

Container created

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

AddedInterface

Add eth0 [10.134.0.164/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

Created

Container created
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

Unhealthy

Startup probe failed: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

ProbeError

Startup probe error: Get "http://10.133.0.213:15021/healthz/ready": dial tcp 10.133.0.213:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

Unhealthy

Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

ProbeError

Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused body:

opendatahub

multus

test-curl-7695-39603a

AddedInterface

Add eth0 [10.132.0.163/23] from ovn-kubernetes

opendatahub

kubelet

test-curl-7695-39603a

Created

Container created

opendatahub

kubelet

test-curl-7695-39603a

Started

Container started

opendatahub

kubelet

test-curl-7695-39603a

Pulled

Container image "registry.access.redhat.com/ubi9/ubi-minimal:9.5" already present on machine and can be accessed by the pod

opendatahub

kubelet

test-curl-7695-39603a

Killing

Stopping container test-curl-7695-39603a

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 2 to 4

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 4; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Created

Container created

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

AddedInterface

Add eth0 [10.133.0.214/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

AddedInterface

Add eth0 [10.134.0.165/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Unhealthy

Startup probe failed: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

ProbeError

Startup probe error: Get "http://10.133.0.214:15021/healthz/ready": dial tcp 10.133.0.214:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Unhealthy

Startup probe failed: Get "http://10.133.0.214:15021/healthz/ready": dial tcp 10.133.0.214:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

ProbeError

Startup probe error: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-mt-c831a4c0-66fbbdcdf5-lsklb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-mt-c831a4c0-6fdb86cf8f-mzsr8

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-extmod-a-25719d-openshift-default-77fc995d65-h5crw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l5fdt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-jcsgg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-d6d9t

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-zxrpk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-28g7h

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-k28nb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-8q7d5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-b6x22

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-l294g

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-mt-c831a4c0-openshift-default-cbd6b4ddf-p2qt6

Killing

Stopping container istio-proxy

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 5; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

AddedInterface

Add eth0 [10.132.0.167/23] from ovn-kubernetes

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 4 to 5

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-25719d-66ffb897b7-9n4pr

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Created

Container created
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-25719d-78689cc99f-2kl4v

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

Killing

Stopping container payload-processing
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

ProbeError

Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Unhealthy

Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w0-302ba0-6879bdc87c-qqkcj

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w0-302ba0-77cd8985c9-fs5wb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-1ff1dc-7fc785c587-zk2dw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-1ff1dc-8574466fb7-w5rsl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-lmc96

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-a-w0-302ba0-openshift-default-5fd7ccb9ff-7zxhk

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

Killing

Stopping container payload-processing

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

AddedInterface

Add eth0 [10.133.0.224/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

AddedInterface

Add eth0 [10.133.0.225/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9hv44

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9hv44

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9hv44

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-9hv44

AddedInterface

Add eth0 [10.132.0.169/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

Created

Container created
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-a-caf74d-74b5f5bc75-9swk9

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-extmod-a-caf74d-55b6d48fb9-4mrdq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9hv44

Unhealthy

Startup probe failed: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9hv44

ProbeError

Startup probe error: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

Unhealthy

Startup probe failed: Get "http://10.133.0.224:15021/healthz/ready": dial tcp 10.133.0.224:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

ProbeError

Startup probe error: Get "http://10.133.0.224:15021/healthz/ready": dial tcp 10.133.0.224:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

ProbeError

Startup probe error: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused body:
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

Unhealthy

Startup probe failed: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused

openshift-ingress

kubelet

e2e-extmod-a-caf74d-openshift-default-6bd6bf9669-7pnqf

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-1ff1dc-openshift-default-6b77ccc689-t9rhq

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w0-6ba6e5-748d8b57c7-n49w5

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 6 to 10

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

AddedInterface

Add eth0 [10.133.0.228/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

AddedInterface

Add eth0 [10.133.0.227/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

AddedInterface

Add eth0 [10.132.0.170/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

AddedInterface

Add eth0 [10.134.0.166/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w0-6ba6e5-5758557f59-tp7lw

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-extmod-b-581487-7cb86cd96c-w2dpt

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-extmod-b-581487-f88795567-8kgw2

Unhealthy

Readiness probe failed: timeout: failed to connect service "10.132.0.127:50052" within 1s: context deadline exceeded
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

Unhealthy

Startup probe failed: Get "http://10.133.0.228:15021/healthz/ready": dial tcp 10.133.0.228:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

ProbeError

Startup probe error: Get "http://10.133.0.227:15021/healthz/ready": dial tcp 10.133.0.227:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

Unhealthy

Startup probe failed: Get "http://10.133.0.227:15021/healthz/ready": dial tcp 10.133.0.227:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

Unhealthy

Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused
(x8)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

ProbeError

Startup probe error: Get "http://10.133.0.228:15021/healthz/ready": dial tcp 10.133.0.228:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

Unhealthy

Startup probe failed: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

ProbeError

Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

ProbeError

Startup probe error: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w2-8ff537-5d69f665cc-hd8bd

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w2-8ff537-7d88cbcd67-r8j8d

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-shared-b-w0-6ba6e5-openshift-default-56cbff86ff-hrkr5

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-extmod-b-581487-openshift-default-755f5b8d9c-b5687

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w2-8ff537-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

multus

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

AddedInterface

Add eth0 [10.132.0.171/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

Created

Container created
(x2)

openshift-ingress

deployment-controller

e2e-worker-w2-8ff537-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9 from 1 to 2

openshift-ingress

replicaset-controller

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9

SuccessfulCreate

Created pod: e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

Started

Container started

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

FailedToRetrieveImagePullSecret

Unable to retrieve some image pull secrets (e2e-worker-w2-8ff537-openshift-default-dockercfg-sbjxb); attempting to pull the image may not succeed.

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

Unhealthy

Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-q7hkr

ProbeError

Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

Unhealthy

Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w2-8ff537-openshift-default-69fb5bbbc9-zq6dz

ProbeError

Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused body:
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w4-190724-65b8997649-4kvmb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w4-190724-8bb994975-5nxgq

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-w0-3ba2f3-5b69fb897-6bmzb

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w0-3ba2f3-5579b8f879-nkpww

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

AddedInterface

Add eth0 [10.134.0.167/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc

SuccessfulCreate

Created pod: e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w4-190724-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-w4-190724-openshift-default-58fbc9d9bc

SuccessfulCreate

Created pod: e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9
(x2)

openshift-ingress

deployment-controller

e2e-worker-w4-190724-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w4-190724-openshift-default-58fbc9d9bc from 1 to 2

openshift-ingress

multus

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

AddedInterface

Add eth0 [10.132.0.172/23] from ovn-kubernetes

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-w0-3ba2f3-openshift-default

SuccessfulRescale

New size: 7; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

deployment-controller

e2e-worker-w0-3ba2f3-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc from 5 to 7

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

Created

Container created

openshift-ingress

multus

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

AddedInterface

Add eth0 [10.134.0.168/23] from ovn-kubernetes

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Created

Container created

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Started

Container started

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

Started

Container started

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

Started

Container started
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

ProbeError

Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

Unhealthy

Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

ProbeError

Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

ProbeError

Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

Unhealthy

Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Unhealthy

Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-ljmk9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w4-190724-openshift-default-58fbc9d9bc-gw9sw

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-v768j

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-7khh8

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8crkg

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-tjn2b

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-8klrb

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-5dwv9

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

ProbeError

Readiness probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused body:

openshift-ingress

kubelet

e2e-worker-w0-3ba2f3-openshift-default-6db4fbcdbc-9r5pt

Unhealthy

Readiness probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-worker-w3-28c598-55548bdb55-j92v2

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-w3-28c598-69f558c87f-h7kvb

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-worker-w3-28c598-openshift-default-7f67789789-ql6gh

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-a-w3-2e0116-openshift-default-65b758dfff-59hsp

Killing

Stopping container istio-proxy
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-a-w3-2e0116-698dfd9db5-s5px6

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-a-w3-2e0116-579d8c6cc9-v6h6v

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-shared-b-w3-51aa8f-openshift-default-7f4595c85f-zqvss

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body:
(x2)

openshift-ingress

kubelet

payload-processing-e2e-shared-b-w3-51aa8f-5968bc585c-vt9ct

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-shared-b-w3-51aa8f-56555b749f-z9fxg

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

gateway_labeler_controller

e2e-worker-main-976689

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-main-976689

openshift-ingress

replicaset-controller

e2e-worker-main-976689-openshift-default-5db554f479

SuccessfulCreate

Created pod: e2e-worker-main-976689-openshift-default-5db554f479-mgddl

openshift-ingress

deployment-controller

e2e-worker-main-976689-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-main-976689-openshift-default-5db554f479 from 0 to 1

openshift-ingress

multus

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

AddedInterface

Add eth0 [10.132.0.173/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-worker-main-976689

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-main-976689-848c4986d9 from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-main-976689

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-main-976689-68f8fb7887 from 0 to 1
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-main-976689-68f8fb7887

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-main-976689-68f8fb7887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-main-976689: serviceaccount "payload-processing-e2e-worker-main-976689" not found
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-main-976689-848c4986d9

FailedCreate

Error creating: pods "payload-processing-e2e-worker-main-976689-848c4986d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-main-976689: serviceaccount "payload-processing-e2e-worker-main-976689" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

Created

Container created

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-main-976689-848c4986d9

SuccessfulCreate

Created pod: payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

openshift-ingress

multus

payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

AddedInterface

Add eth0 [10.132.0.175/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-main-976689-68f8fb7887

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

openshift-ingress

multus

payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

AddedInterface

Add eth0 [10.132.0.174/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x5)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

ProbeError

Startup probe error: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

Unhealthy

Startup probe failed: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9hv44

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-xqsk9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-j4f5x

Killing

Stopping container istio-proxy

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-j4f5x

openshift-ingress

replicaset-controller

maas-default-gateway-openshift-default-7d9897b54c

SuccessfulDelete

Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-sjcrx
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 3; reason: All metrics below target

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7x5qn

Killing

Stopping container istio-proxy

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 10 to 3

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-sjcrx

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8c8g9

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-zq7gr

Killing

Stopping container istio-proxy
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-main-976689-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x4)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-main-976689-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

opendatahub

replicaset-controller

maas-controller-64d475bcc9

SuccessfulDelete

Deleted pod: maas-controller-64d475bcc9-z7whx

opendatahub

kubelet

maas-controller-64d475bcc9-z7whx

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-64d475bcc9

SuccessfulCreate

Created pod: maas-controller-64d475bcc9-m8vd5

opendatahub

multus

maas-controller-64d475bcc9-m8vd5

AddedInterface

Add eth0 [10.133.0.239/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-64d475bcc9-m8vd5

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159"

opendatahub

kubelet

maas-controller-64d475bcc9-m8vd5

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159" in 898ms (898ms including waiting). Image size: 200094742 bytes.

opendatahub

kubelet

maas-controller-64d475bcc9-m8vd5

Created

Container created

opendatahub

kubelet

maas-controller-64d475bcc9-m8vd5

Started

Container started
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 6; reason: cpu resource utilization (percentage of request) above target
(x3)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 3 to 6

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-hjj86

AddedInterface

Add eth0 [10.133.0.240/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-krv87

AddedInterface

Add eth0 [10.132.0.177/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

AddedInterface

Add eth0 [10.133.0.241/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

opendatahub

maas-controller-64d475bcc9-m8vd5_46adea9b-f638-4c6b-a29d-bc0a17e1d4cd

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-64d475bcc9-m8vd5_46adea9b-f638-4c6b-a29d-bc0a17e1d4cd became leader
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Unhealthy

Startup probe failed: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

Unhealthy

Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

ProbeError

Startup probe error: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

ProbeError

Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

ProbeError

Startup probe error: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused body:
(x7)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Unhealthy

Startup probe failed: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

Unhealthy

Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

ProbeError

Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

ProbeError

Startup probe error: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Unhealthy

Startup probe failed: Get "http://10.133.0.241:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Unhealthy

Startup probe failed: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

ProbeError

Startup probe error: Get "http://10.133.0.241:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 8; reason: All metrics below target

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 6 to 8

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

Created

Container created

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

AddedInterface

Add eth0 [10.133.0.243/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-vktw8

AddedInterface

Add eth0 [10.132.0.178/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

Started

Container started
(x2)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled down replica set maas-controller-64d475bcc9 from 1 to 0

opendatahub

replicaset-controller

maas-controller-64d475bcc9

SuccessfulDelete

Deleted pod: maas-controller-64d475bcc9-m8vd5

opendatahub

kubelet

maas-controller-64d475bcc9-m8vd5

Killing

Stopping container manager

opendatahub

replicaset-controller

maas-controller-64d475bcc9

SuccessfulCreate

Created pod: maas-controller-64d475bcc9-cs8xr

opendatahub

multus

maas-controller-64d475bcc9-cs8xr

AddedInterface

Add eth0 [10.133.0.244/23] from ovn-kubernetes

opendatahub

kubelet

maas-controller-64d475bcc9-cs8xr

Pulling

Pulling image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159"
(x3)

opendatahub

deployment-controller

maas-controller

ScalingReplicaSet

Scaled up replica set maas-controller-64d475bcc9 from 0 to 1

opendatahub

kubelet

maas-controller-64d475bcc9-cs8xr

Started

Container started

opendatahub

kubelet

maas-controller-64d475bcc9-cs8xr

Created

Container created

opendatahub

kubelet

maas-controller-64d475bcc9-cs8xr

Pulled

Successfully pulled image "quay.io/opendatahub/maas-controller@sha256:a426001269a766a3c983788821946a92720ad289bbb0fe6f48ef45b1229db159" in 974ms (974ms including waiting). Image size: 200094742 bytes.
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

ProbeError

Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

ProbeError

Startup probe error: Get "http://10.133.0.243:15021/healthz/ready": dial tcp 10.133.0.243:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

Unhealthy

Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

Unhealthy

Startup probe failed: Get "http://10.133.0.243:15021/healthz/ready": dial tcp 10.133.0.243:15021: connect: connection refused

opendatahub

maas-controller-64d475bcc9-cs8xr_794e8b90-76d7-424f-acf8-c0e821bdd481

maas-controller.models-as-a-service.opendatahub.io

LeaderElection

maas-controller-64d475bcc9-cs8xr_794e8b90-76d7-424f-acf8-c0e821bdd481 became leader

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulDelete

Deleted pod: kuadrant-operator-controller-manager-678d87766f-jpshj

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled down replica set kuadrant-operator-controller-manager-678d87766f from 1 to 0

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-jpshj

Killing

Stopping container manager

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

NeedsReinstall

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

AllRequirementsMet

all requirements found, attempting install
(x3)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

waiting for install components to report healthy
(x2)

kuadrant-system

deployment-controller

kuadrant-operator-controller-manager

ScalingReplicaSet

Scaled up replica set kuadrant-operator-controller-manager-678d87766f from 0 to 1

kuadrant-system

replicaset-controller

kuadrant-operator-controller-manager-678d87766f

SuccessfulCreate

Created pod: kuadrant-operator-controller-manager-678d87766f-b9zd5

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

ComponentUnhealthy

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallWaiting

installing: waiting for deployment kuadrant-operator-controller-manager to become ready: deployment "kuadrant-operator-controller-manager" not available: Deployment does not have minimum availability.

kuadrant-system

multus

kuadrant-operator-controller-manager-678d87766f-b9zd5

AddedInterface

Add eth0 [10.134.0.171/23] from ovn-kubernetes

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-b9zd5

Created

Container created

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-b9zd5

Started

Container started

kuadrant-system

kubelet

kuadrant-operator-controller-manager-678d87766f-b9zd5

Pulled

Container image "registry.redhat.io/rhcl-1/rhcl-rhel9-operator@sha256:cc2471e083a549afb80327aa87e1d32f05e40561c3f82275b0cdb88af5ee5e8e" already present on machine and can be accessed by the pod
(x2)

kuadrant-system

operator-lifecycle-manager

rhcl-operator.v1.4.3

InstallSucceeded

install strategy completed with no errors

kuadrant-system

kuadrant-operator-controller-manager-678d87766f-b9zd5_c6ab287a-248d-427f-90b6-e7eb1f51a924

f139389e.kuadrant.io

LeaderElection

kuadrant-operator-controller-manager-678d87766f-b9zd5_c6ab287a-248d-427f-90b6-e7eb1f51a924 became leader
(x4)

openshift-ingress

horizontal-pod-autoscaler

maas-default-gateway-openshift-default

SuccessfulRescale

New size: 10; reason: cpu resource utilization (percentage of request) above target
(x2)

openshift-ingress

deployment-controller

maas-default-gateway-openshift-default

ScalingReplicaSet

Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 8 to 10

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

Created

Container created

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

Started

Container started

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-9tp28

AddedInterface

Add eth0 [10.132.0.179/23] from ovn-kubernetes

openshift-ingress

multus

maas-default-gateway-openshift-default-7d9897b54c-2np4m

AddedInterface

Add eth0 [10.134.0.172/23] from ovn-kubernetes

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

Started

Container started

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

Created

Container created
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

Unhealthy

Startup probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

ProbeError

Startup probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

ProbeError

Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused body:
(x6)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

Unhealthy

Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused

openshift-ingress

kubelet

payload-processing-e2e-worker-main-976689-848c4986d9-x4sx4

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-main-976689-68f8fb7887-qqdnt

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

e2e-worker-main-976689-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-main-976689-openshift-default-5db554f479 from 1 to 2

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-main-976689-openshift-default

SuccessfulRescale

New size: 2; reason: cpu resource utilization (percentage of request) above target

openshift-ingress

replicaset-controller

e2e-worker-main-976689-openshift-default-5db554f479

SuccessfulCreate

Created pod: e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

Created

Container created

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

Started

Container started

openshift-ingress

multus

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

AddedInterface

Add eth0 [10.133.1.4/23] from ovn-kubernetes
(x5)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

ProbeError

Startup probe error: Get "http://10.133.1.4:15021/healthz/ready": dial tcp 10.133.1.4:15021: connect: connection refused body:
(x5)

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

Unhealthy

Startup probe failed: Get "http://10.133.1.4:15021/healthz/ready": dial tcp 10.133.1.4:15021: connect: connection refused

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-mgddl

Killing

Stopping container istio-proxy

openshift-ingress

kubelet

e2e-worker-main-976689-openshift-default-5db554f479-nkwj5

Killing

Stopping container istio-proxy

openshift-ingress

gateway_labeler_controller

e2e-ait-501c44d6

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-ait-501c44d6

openshift-ingress

deployment-controller

e2e-ait-501c44d6-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-ait-501c44d6-openshift-default-74b687d8f5 from 0 to 1

openshift-ingress

replicaset-controller

e2e-ait-501c44d6-openshift-default-74b687d8f5

SuccessfulCreate

Created pod: e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

openshift-ingress

multus

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

AddedInterface

Add eth0 [10.132.0.180/23] from ovn-kubernetes

openshift-ingress

deployment-controller

payload-processing-e2e-ait-501c44d6

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-ait-501c44d6-5d8fc4b77 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-501c44d6-5d8fc4b77

SuccessfulCreate

Created pod: payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-ait-501c44d6-5d8fc4b77

FailedCreate

Error creating: pods "payload-processing-e2e-ait-501c44d6-5d8fc4b77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-501c44d6: serviceaccount "payload-processing-e2e-ait-501c44d6" not found
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-501c44d6-58df7fb659

FailedCreate

Error creating: pods "payload-pre-processing-e2e-ait-501c44d6-58df7fb659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-501c44d6: serviceaccount "payload-processing-e2e-ait-501c44d6" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-ait-501c44d6

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-ait-501c44d6-58df7fb659 from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-ait-501c44d6-58df7fb659

SuccessfulCreate

Created pod: payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

openshift-ingress

multus

payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl

AddedInterface

Add eth0 [10.132.0.182/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl

Created

Container created

openshift-ingress

kubelet

payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

AddedInterface

Add eth0 [10.132.0.181/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Started

Container started
(x2)

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Created

Container created

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-501c44d6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-501c44d6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)
(x6)

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Unhealthy

Startup probe failed: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused
(x6)

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

ProbeError

Startup probe error: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused body:

openshift-ingress

kubelet

payload-processing-e2e-ait-501c44d6-5d8fc4b77-sf2gl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-ait-501c44d6-58df7fb659-jxhwc

Killing

Stopping container payload-pre-processing

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Killing

Stopping container istio-proxy
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-501c44d6-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x3)

openshift-ingress

horizontal-pod-autoscaler

e2e-ait-501c44d6-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

ProbeError

Readiness probe error: HTTP probe failed with statuscode: 503 body: upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused

openshift-ingress

kubelet

e2e-ait-501c44d6-openshift-default-74b687d8f5-pvq44

Unhealthy

Readiness probe failed: HTTP probe failed with statuscode: 503

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.Secret llm/e2e-prio-pool-7a816b61-kserve-self-signed-certs

llm

kubelet

e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

Started

Container started

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.Service llm/e2e-prio-pool-7a816b61-kserve-workload-svc

llm

deployment-controller

e2e-prio-pool-7a816b61-kserve-router-scheduler

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6 from 0 to 1

llm

deployment-controller

e2e-prio-pool-7a816b61-kserve

ScalingReplicaSet

Scaled up replica set e2e-prio-pool-7a816b61-kserve-5594ff5697 from 0 to 1

llm

multus

e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

AddedInterface

Add eth0 [10.134.0.173/23] from ovn-kubernetes

llm

kubelet

e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

Created

Container created

llm

replicaset-controller

e2e-prio-pool-7a816b61-kserve-5594ff5697

SuccessfulCreate

Created pod: e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

llm

replicaset-controller

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6

SuccessfulCreate

Created pod: e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.Service llm/e2e-prio-pool-7a816b61-epp-service

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.Deployment llm/e2e-prio-pool-7a816b61-kserve

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.ServiceAccount llm/e2e-prio-pool-7a816b61-epp-sa

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.ClusterRoleBinding /llm-e2e-prio-pool-7a816b61-epp-auth-rb

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.Role llm/e2e-prio-pool-7a816b61-epp-role

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.RoleBinding llm/e2e-prio-pool-7a816b61-epp-rb

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

Created v1.Deployment llm/e2e-prio-pool-7a816b61-kserve-router-scheduler

llm

kubelet

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Pulling

Pulling image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827"

llm

multus

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

AddedInterface

Add eth0 [10.132.0.184/23] from ovn-kubernetes

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Updated

Updated v1.Secret llm/e2e-prio-pool-7a816b61-kserve-self-signed-certs
(x9)

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Created

(combined from similar events): Created v1.DestinationRule llm/e2e-prio-pool-7a816b61-kserve-shadow-svc

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Updated

Updated v1.HTTPRoute llm/e2e-prio-pool-7a816b61-kserve-route

llm

kubelet

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Pulled

Successfully pulled image "quay.io/opendatahub/odh-llm-d-router-endpoint-picker@sha256:5fe24a45d9c8039b6db33e8be331dab766610e6eb60f9d2bec1ecc1153ca8827" in 4.944s (4.944s including waiting). Image size: 184206420 bytes.

llm

kubelet

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Created

Container created

llm

kubelet

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Started

Container started

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

LLMInferenceServiceReady

LLMInferenceService [e2e-prio-pool-7a816b61] is Ready

llm

LLMInferenceServiceController

e2e-prio-pool-7a816b61

Deleted

Deleted v1.ClusterRoleBinding /llm-e2e-prio-pool-7a816b61-epp-auth-rb

llm

kubelet

e2e-prio-pool-7a816b61-kserve-5594ff5697-ksmxr

Killing

Stopping container main

llm

kubelet

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Killing

Stopping container main

llm

deployment-controller

e2e-same-id-a-fff8f9af-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-a-fff8f9af-kserve-775d49d8b9 from 0 to 1

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.DestinationRule llm/e2e-same-id-a-fff8f9af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.HTTPRoute llm/e2e-same-id-a-fff8f9af-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.Service llm/e2e-same-id-a-fff8f9af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.Deployment llm/e2e-same-id-a-fff8f9af-kserve

llm

replicaset-controller

e2e-same-id-a-fff8f9af-kserve-775d49d8b9

SuccessfulCreate

Created pod: e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.Secret llm/e2e-same-id-a-fff8f9af-kserve-self-signed-certs

llm

kubelet

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

Created

Container created

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.PodMonitor llm/e2e-same-id-a-fff8f9af-kserve-llmisvc-engine

llm

multus

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

AddedInterface

Add eth0 [10.134.0.174/23] from ovn-kubernetes

llm

kubelet

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

Started

Container started

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.NetworkPolicy llm/e2e-same-id-a-fff8f9af-prometheus-scraping

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Created

Created v1.PodMonitor llm/e2e-same-id-a-fff8f9af-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

Updated

Updated v1.Secret llm/e2e-same-id-a-fff8f9af-kserve-self-signed-certs
(x4)

llm

kubelet

e2e-prio-pool-7a816b61-kserve-router-scheduler-cccd88cf6-4k8md

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-a-fff8f9af] is Ready

llm

deployment-controller

e2e-same-id-b-fff8f9af-kserve

ScalingReplicaSet

Scaled up replica set e2e-same-id-b-fff8f9af-kserve-6cf9476949 from 0 to 1

llm

replicaset-controller

e2e-same-id-b-fff8f9af-kserve-6cf9476949

SuccessfulCreate

Created pod: e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.Service llm/e2e-same-id-b-fff8f9af-kserve-workload-svc

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.HTTPRoute llm/e2e-same-id-b-fff8f9af-kserve-route

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.Deployment llm/e2e-same-id-b-fff8f9af-kserve

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.Secret llm/e2e-same-id-b-fff8f9af-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.DestinationRule llm/e2e-same-id-b-fff8f9af-kserve-workload-svc

llm

kubelet

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

Pulled

Container image "ghcr.io/llm-d/llm-d-inference-sim@sha256:c3ba435081a4d032676b218ea34eb3a1c54507da0fade2f6297f9c37894fe0d1" already present on machine and can be accessed by the pod

llm

kubelet

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

Created

Container created

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.PodMonitor llm/e2e-same-id-b-fff8f9af-kserve-llmisvc-engine-default

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.PodMonitor llm/e2e-same-id-b-fff8f9af-kserve-llmisvc-engine

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Created

Created v1.NetworkPolicy llm/e2e-same-id-b-fff8f9af-prometheus-scraping

llm

multus

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

AddedInterface

Add eth0 [10.132.0.185/23] from ovn-kubernetes

llm

kubelet

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

Started

Container started

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

Updated

Updated v1.Secret llm/e2e-same-id-b-fff8f9af-kserve-self-signed-certs

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

LLMInferenceServiceReady

LLMInferenceService [e2e-same-id-b-fff8f9af] is Ready
(x6)

llm

LLMInferenceServiceController

e2e-same-id-b-fff8f9af

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-a-fff8f9af in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-b-fff8f9af; safe to ignore if traffic splitting is planned
(x2)

llm

LLMInferenceServiceController

e2e-same-id-a-fff8f9af

ModelNameCollision

model name "test/e2e-same-model-id" overlaps with e2e-same-id-b-fff8f9af in namespace llm; shared publisher paths and model-routing headers cause the gateway to shadow one service. Reachable at its own address llm/e2e-same-id-a-fff8f9af; safe to ignore if traffic splitting is planned

llm

kubelet

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

Killing

Stopping container main

llm

kubelet

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

Killing

Stopping container main

llm

kubelet

e2e-same-id-a-fff8f9af-kserve-775d49d8b9-dzrg4

Unhealthy

Readiness probe failed: Get "https://10.134.0.174:8000/ready": dial tcp 10.134.0.174:8000: connect: connection refused

llm

kubelet

e2e-same-id-b-fff8f9af-kserve-6cf9476949-rh2hf

Unhealthy

Readiness probe failed: Get "https://10.132.0.185:8000/ready": dial tcp 10.132.0.185:8000: connect: connection refused
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

Unhealthy

Readiness probe failed: Get "http://10.132.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qpmzx

ProbeError

Readiness probe error: Get "http://10.132.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

replicaset-controller

payload-pre-processing-56f8865864

SuccessfulCreate

Created pod: payload-pre-processing-56f8865864-tcnsn

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-56f8865864 from 0 to 1

openshift-ingress

kubelet

payload-pre-processing-56f8865864-tcnsn

Created

Container created

openshift-ingress

kubelet

payload-pre-processing-56f8865864-tcnsn

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-56f8865864-tcnsn

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-pre-processing-56f8865864-tcnsn

AddedInterface

Add eth0 [10.132.0.186/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-7b98bb678c-7dngs

Killing

Stopping container payload-pre-processing

openshift-ingress

replicaset-controller

payload-pre-processing-7b98bb678c

SuccessfulDelete

Deleted pod: payload-pre-processing-7b98bb678c-7dngs

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-7b98bb678c from 1 to 0

openshift-ingress

replicaset-controller

payload-processing-54ddd7df84

SuccessfulCreate

Created pod: payload-processing-54ddd7df84-r5p2d

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-54ddd7df84 from 0 to 1

openshift-ingress

multus

payload-processing-54ddd7df84-r5p2d

AddedInterface

Add eth0 [10.132.0.187/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-processing-54ddd7df84-r5p2d

Created

Container created

openshift-ingress

kubelet

payload-processing-54ddd7df84-r5p2d

Started

Container started

openshift-ingress

kubelet

payload-processing-54ddd7df84-r5p2d

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-55fbbbf9dc from 1 to 0

openshift-ingress

kubelet

payload-processing-55fbbbf9dc-q2fb6

Killing

Stopping container payload-processing

openshift-ingress

replicaset-controller

payload-processing-55fbbbf9dc

SuccessfulDelete

Deleted pod: payload-processing-55fbbbf9dc-q2fb6

openshift-ingress

replicaset-controller

payload-pre-processing-5b4cf47bfb

SuccessfulCreate

Created pod: payload-pre-processing-5b4cf47bfb-xjj8z

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled up replica set payload-processing-6759564b6c from 0 to 1

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled up replica set payload-pre-processing-5b4cf47bfb from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-6759564b6c

SuccessfulCreate

Created pod: payload-processing-6759564b6c-wpg92

openshift-ingress

kubelet

payload-pre-processing-5b4cf47bfb-xjj8z

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-6759564b6c-wpg92

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-5b4cf47bfb-xjj8z

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-5b4cf47bfb-xjj8z

Created

Container created

openshift-ingress

kubelet

payload-processing-6759564b6c-wpg92

Created

Container created

openshift-ingress

kubelet

payload-processing-6759564b6c-wpg92

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

multus

payload-processing-6759564b6c-wpg92

AddedInterface

Add eth0 [10.132.0.188/23] from ovn-kubernetes

openshift-ingress

multus

payload-pre-processing-5b4cf47bfb-xjj8z

AddedInterface

Add eth0 [10.132.0.189/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

payload-pre-processing-56f8865864

SuccessfulDelete

Deleted pod: payload-pre-processing-56f8865864-tcnsn

openshift-ingress

kubelet

payload-pre-processing-56f8865864-tcnsn

Killing

Stopping container payload-pre-processing

openshift-ingress

deployment-controller

payload-processing

ScalingReplicaSet

Scaled down replica set payload-processing-54ddd7df84 from 1 to 0

openshift-ingress

replicaset-controller

payload-processing-54ddd7df84

SuccessfulDelete

Deleted pod: payload-processing-54ddd7df84-r5p2d

openshift-ingress

kubelet

payload-processing-54ddd7df84-r5p2d

Killing

Stopping container payload-processing

openshift-ingress

deployment-controller

payload-pre-processing

ScalingReplicaSet

Scaled down replica set payload-pre-processing-56f8865864 from 1 to 0

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

SuccessfulCreate

Created pod: payload-processing-external-model-5bcd7997-lbz5s

models-as-a-service

deployment-controller

payload-processing-external-model

ScalingReplicaSet

Scaled up replica set payload-processing-external-model-5bcd7997 from 0 to 1
(x7)

models-as-a-service

replicaset-controller

payload-processing-external-model-5bcd7997

FailedCreate

Error creating: pods "payload-processing-external-model-5bcd7997-" is forbidden: error looking up service account models-as-a-service/payload-processing-external-model: serviceaccount "payload-processing-external-model" not found

models-as-a-service

multus

payload-processing-external-model-5bcd7997-lbz5s

AddedInterface

Add eth0 [10.132.0.190/23] from ovn-kubernetes

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-lbz5s

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-lbz5s

Created

Container created

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-lbz5s

Started

Container started

models-as-a-service

kubelet

payload-processing-external-model-5bcd7997-lbz5s

Killing

Stopping container payload-processing
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

ProbeError

Readiness probe error: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x2)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-hjj86

Unhealthy

Readiness probe failed: Get "http://10.133.0.240:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

ProbeError

Readiness probe error: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x4)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-krv87

Unhealthy

Readiness probe failed: Get "http://10.132.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

Unhealthy

Readiness probe failed: Get "http://10.133.0.241:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-qb2mm

ProbeError

Readiness probe error: Get "http://10.133.0.241:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

gateway_labeler_controller

e2e-worker-single-main-2624b0

AddedLabel

Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-2624b0

openshift-ingress

deployment-controller

e2e-worker-single-main-2624b0-openshift-default

ScalingReplicaSet

Scaled up replica set e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49 from 0 to 1

openshift-ingress

multus

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

AddedInterface

Add eth0 [10.132.0.191/23] from ovn-kubernetes

openshift-ingress

replicaset-controller

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49

SuccessfulCreate

Created pod: e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

openshift-ingress

deployment-controller

payload-processing-e2e-worker-single-main-2624b0

ScalingReplicaSet

Scaled up replica set payload-processing-e2e-worker-single-main-2624b0-fc9d6d767 from 0 to 1

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767

SuccessfulCreate

Created pod: payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl
(x7)

openshift-ingress

replicaset-controller

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767

FailedCreate

Error creating: pods "payload-processing-e2e-worker-single-main-2624b0-fc9d6d767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2624b0: serviceaccount "payload-processing-e2e-worker-single-main-2624b0" not found

openshift-ingress

deployment-controller

payload-pre-processing-e2e-worker-single-main-2624b0

ScalingReplicaSet

Scaled up replica set payload-pre-processing-e2e-worker-single-main-2624b0-57979ddddc from 0 to 1

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-2624b0-57979ddddc

SuccessfulCreate

Created pod: payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx
(x6)

openshift-ingress

replicaset-controller

payload-pre-processing-e2e-worker-single-main-2624b0-57979ddddc

FailedCreate

Error creating: pods "payload-pre-processing-e2e-worker-single-main-2624b0-57979ddddc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-2624b0: serviceaccount "payload-processing-e2e-worker-single-main-2624b0" not found

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

Started

Container started

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

Created

Container created

openshift-ingress

multus

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

AddedInterface

Add eth0 [10.132.0.192/23] from ovn-kubernetes

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

Pulled

Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

Started

Container started

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

Created

Container created

openshift-ingress

multus

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

AddedInterface

Add eth0 [10.132.0.193/23] from ovn-kubernetes
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

Pulled

Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

Created

Container created
(x2)

openshift-ingress

kubelet

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

Started

Container started
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-2624b0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x2)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-2624b0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

Unhealthy

Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused
(x5)

openshift-ingress

kubelet

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

ProbeError

Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused body:

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-2624b0-openshift-default

FailedComputeMetricsReplicas

invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

horizontal-pod-autoscaler

e2e-worker-single-main-2624b0-openshift-default

FailedGetResourceMetric

failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

Killing

Stopping container payload-processing

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

Killing

Stopping container payload-pre-processing
(x2)

openshift-ingress

kubelet

payload-pre-processing-e2e-worker-single-main-2624b0-57979wndqx

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")
(x2)

openshift-ingress

kubelet

payload-processing-e2e-worker-single-main-2624b0-fc9d6d767lg2hl

Unhealthy

Readiness probe failed: service unhealthy (responded with "NOT_SERVING")

openshift-ingress

kubelet

e2e-worker-single-main-2624b0-openshift-default-5dc45cfc49wjspj

Killing

Stopping container istio-proxy
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

ProbeError

Readiness probe error: Get "http://10.133.0.243:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-pn2tc

Unhealthy

Readiness probe failed: Get "http://10.133.0.243:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

ProbeError

Readiness probe error: Get "http://10.132.0.178:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-vktw8

Unhealthy

Readiness probe failed: Get "http://10.132.0.178:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

ProbeError

Readiness probe error: Get "http://10.134.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-7ldgk

Unhealthy

Readiness probe failed: Get "http://10.134.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

ProbeError

Readiness probe error: Get "http://10.134.0.23:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-8ckbd

Unhealthy

Readiness probe failed: Get "http://10.134.0.23:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

ProbeError

Readiness probe error: Get "http://10.132.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

ProbeError

Readiness probe error: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) body:
(x3)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-2np4m

Unhealthy

Readiness probe failed: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
(x5)

openshift-ingress

kubelet

maas-default-gateway-openshift-default-7d9897b54c-9tp28

Unhealthy

Readiness probe failed: Get "http://10.132.0.179:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)