=== events in openshift-ingress (2026-10-31T17:12:01Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 59m Normal Scheduled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Successfully assigned openshift-ingress/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk to ip-10-0-129-180.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r to ip-10-0-140-84.us-west-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 to ip-10-0-129-180.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-cpsqt to ip-10-0-129-180.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-dx8zr to ip-10-0-140-84.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-fqh2n to ip-10-0-129-180.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 to ip-10-0-140-84.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x to ip-10-0-140-84.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-cnft9 to ip-10-0-140-84.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-8965v to ip-10-0-140-84.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 to ip-10-0-129-180.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ggxdf to ip-10-0-142-127.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Successfully assigned openshift-ingress/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf to ip-10-0-140-84.us-west-2.compute.internal 66m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl to ip-10-0-142-127.us-west-2.compute.internal 64m Normal Scheduled pod/kube-auth-proxy-744f97f697-pfr86 Successfully assigned openshift-ingress/kube-auth-proxy-744f97f697-pfr86 to ip-10-0-140-84.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Successfully assigned openshift-ingress/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 to ip-10-0-140-84.us-west-2.compute.internal 64m Normal Scheduled pod/kube-auth-proxy-744f97f697-vjlc2 Successfully assigned openshift-ingress/kube-auth-proxy-744f97f697-vjlc2 to ip-10-0-129-180.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q to ip-10-0-142-127.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 to ip-10-0-142-127.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-zgr9j to ip-10-0-129-180.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 to ip-10-0-140-84.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Successfully assigned openshift-ingress/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-vkxlw to ip-10-0-142-127.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-tc5qb to ip-10-0-129-180.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-t7qph to ip-10-0-142-127.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-sr5pb to ip-10-0-142-127.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Successfully assigned openshift-ingress/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc to ip-10-0-142-127.us-west-2.compute.internal 69m Normal Scheduled pod/router-default-746fc4dd59-qfxr9 Successfully assigned openshift-ingress/router-default-746fc4dd59-qfxr9 to ip-10-0-142-127.us-west-2.compute.internal 70m Warning FailedScheduling pod/router-default-746fc4dd59-qfxr9 no nodes available to schedule pods 75m Warning FailedScheduling pod/router-default-746fc4dd59-qfxr9 no nodes available to schedule pods 52m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j to ip-10-0-142-127.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Successfully assigned openshift-ingress/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf to ip-10-0-142-127.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq to ip-10-0-142-127.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 to ip-10-0-142-127.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 to ip-10-0-140-84.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp to ip-10-0-129-180.us-west-2.compute.internal 15m Normal Scheduled pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Successfully assigned openshift-ingress/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv to ip-10-0-140-84.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd to ip-10-0-142-127.us-west-2.compute.internal 6m22s Normal Scheduled pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Successfully assigned openshift-ingress/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht to ip-10-0-142-127.us-west-2.compute.internal 10m Normal Scheduled pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Successfully assigned openshift-ingress/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n to ip-10-0-129-180.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc to ip-10-0-140-84.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Successfully assigned openshift-ingress/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb to ip-10-0-142-127.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls to ip-10-0-140-84.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc to ip-10-0-129-180.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps to ip-10-0-142-127.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Successfully assigned openshift-ingress/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw to ip-10-0-140-84.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s to ip-10-0-140-84.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk to ip-10-0-142-127.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 to ip-10-0-129-180.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c to ip-10-0-140-84.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g to ip-10-0-129-180.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Successfully assigned openshift-ingress/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv to ip-10-0-142-127.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv to ip-10-0-140-84.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Successfully assigned openshift-ingress/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 to ip-10-0-142-127.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff to ip-10-0-142-127.us-west-2.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx to ip-10-0-129-180.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Successfully assigned openshift-ingress/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl to ip-10-0-140-84.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn to ip-10-0-140-84.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj to ip-10-0-129-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x to ip-10-0-142-127.us-west-2.compute.internal 59m Normal Scheduled pod/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh to ip-10-0-129-180.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps to ip-10-0-142-127.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl to ip-10-0-142-127.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Successfully assigned openshift-ingress/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Successfully assigned openshift-ingress/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 to ip-10-0-129-180.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w to ip-10-0-142-127.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Successfully assigned openshift-ingress/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Successfully assigned openshift-ingress/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 to ip-10-0-129-180.us-west-2.compute.internal 15m Normal Scheduled pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch to ip-10-0-140-84.us-west-2.compute.internal 10m Normal Scheduled pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk to ip-10-0-140-84.us-west-2.compute.internal 37m Normal Scheduled pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Successfully assigned openshift-ingress/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l to ip-10-0-142-127.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 to ip-10-0-140-84.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 to ip-10-0-129-180.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk to ip-10-0-140-84.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl to ip-10-0-129-180.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f to ip-10-0-140-84.us-west-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 Successfully assigned openshift-ingress/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 to ip-10-0-140-84.us-west-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx Successfully assigned openshift-ingress/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx to ip-10-0-140-84.us-west-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Successfully assigned openshift-ingress/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk Successfully assigned openshift-ingress/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk to ip-10-0-140-84.us-west-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l to ip-10-0-142-127.us-west-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb to ip-10-0-140-84.us-west-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x Successfully assigned openshift-ingress/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x to ip-10-0-140-84.us-west-2.compute.internal 5m11s Normal Scheduled pod/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 to ip-10-0-140-84.us-west-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv Successfully assigned openshift-ingress/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv to ip-10-0-142-127.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Successfully assigned openshift-ingress/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs to ip-10-0-129-180.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh Successfully assigned openshift-ingress/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh to ip-10-0-140-84.us-west-2.compute.internal 6m14s Normal Scheduled pod/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf Successfully assigned openshift-ingress/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf to ip-10-0-140-84.us-west-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj Successfully assigned openshift-ingress/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 Successfully assigned openshift-ingress/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Successfully assigned openshift-ingress/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln to ip-10-0-129-180.us-west-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Successfully assigned openshift-ingress/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 to ip-10-0-140-84.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-dgfgj to ip-10-0-129-180.us-west-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Successfully assigned openshift-ingress/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt to ip-10-0-129-180.us-west-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Successfully assigned openshift-ingress/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh to ip-10-0-140-84.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Successfully assigned openshift-ingress/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb to ip-10-0-129-180.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 to ip-10-0-129-180.us-west-2.compute.internal 60m Normal Scheduled pod/payload-processing-55fbbbf9dc-9f4p2 Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-9f4p2 to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct to ip-10-0-142-127.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 to ip-10-0-129-180.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Successfully assigned openshift-ingress/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr to ip-10-0-140-84.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 to ip-10-0-142-127.us-west-2.compute.internal 64m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-7c689579c-cjljn Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7c689579c-cjljn to ip-10-0-142-127.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt to ip-10-0-140-84.us-west-2.compute.internal 41m Normal Scheduled pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Successfully assigned openshift-ingress/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 to ip-10-0-142-127.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n to ip-10-0-129-180.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj to ip-10-0-140-84.us-west-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h to ip-10-0-140-84.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Successfully assigned openshift-ingress/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx to ip-10-0-140-84.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Successfully assigned openshift-ingress/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx to ip-10-0-142-127.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Successfully assigned openshift-ingress/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 to ip-10-0-140-84.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq to ip-10-0-129-180.us-west-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Successfully assigned openshift-ingress/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq to ip-10-0-140-84.us-west-2.compute.internal 40m Normal Scheduled pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Successfully assigned openshift-ingress/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 to ip-10-0-142-127.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg to ip-10-0-129-180.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz to ip-10-0-142-127.us-west-2.compute.internal 28m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w to ip-10-0-129-180.us-west-2.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j to ip-10-0-140-84.us-west-2.compute.internal 59m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk to ip-10-0-129-180.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Successfully assigned openshift-ingress/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz to ip-10-0-129-180.us-west-2.compute.internal 49m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm to ip-10-0-140-84.us-west-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Successfully assigned openshift-ingress/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Successfully assigned openshift-ingress/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh to ip-10-0-140-84.us-west-2.compute.internal 66m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-mhsg7 to ip-10-0-140-84.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ghk9c to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb to ip-10-0-140-84.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-h8jrx to ip-10-0-140-84.us-west-2.compute.internal 5m19s Normal Scheduled pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Successfully assigned openshift-ingress/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd to ip-10-0-140-84.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz to ip-10-0-140-84.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-7c9qm to ip-10-0-140-84.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 to ip-10-0-142-127.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh to ip-10-0-140-84.us-west-2.compute.internal 22m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-5tkrq to ip-10-0-129-180.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-jbh6n to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 to ip-10-0-129-180.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw to ip-10-0-129-180.us-west-2.compute.internal 66m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 to ip-10-0-142-127.us-west-2.compute.internal 43m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Successfully assigned openshift-ingress/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-9d8tx to ip-10-0-129-180.us-west-2.compute.internal 44m Normal Scheduled pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Successfully assigned openshift-ingress/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 to ip-10-0-129-180.us-west-2.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-lzm9d to ip-10-0-140-84.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk to ip-10-0-129-180.us-west-2.compute.internal 59m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 to ip-10-0-129-180.us-west-2.compute.internal 63m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-ndttp to ip-10-0-140-84.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 to ip-10-0-142-127.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-m4gjr to ip-10-0-142-127.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-m7bsq to ip-10-0-142-127.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-mb94r to ip-10-0-129-180.us-west-2.compute.internal 31m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-mqgxj to ip-10-0-140-84.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 to ip-10-0-140-84.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-nbw47 to ip-10-0-140-84.us-west-2.compute.internal 58m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q to ip-10-0-129-180.us-west-2.compute.internal 15m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb to ip-10-0-140-84.us-west-2.compute.internal 10m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth to ip-10-0-140-84.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj to ip-10-0-140-84.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 to ip-10-0-129-180.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc to ip-10-0-140-84.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f to ip-10-0-142-127.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 to ip-10-0-140-84.us-west-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n to ip-10-0-129-180.us-west-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 to ip-10-0-140-84.us-west-2.compute.internal 46m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr to ip-10-0-142-127.us-west-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp to ip-10-0-140-84.us-west-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr to ip-10-0-142-127.us-west-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz to ip-10-0-140-84.us-west-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj to ip-10-0-140-84.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh to ip-10-0-140-84.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc to ip-10-0-129-180.us-west-2.compute.internal 5m12s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb to ip-10-0-142-127.us-west-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 to ip-10-0-140-84.us-west-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 to ip-10-0-140-84.us-west-2.compute.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 to ip-10-0-140-84.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q to ip-10-0-140-84.us-west-2.compute.internal 6m14s Normal Scheduled pod/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c to ip-10-0-140-84.us-west-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 to ip-10-0-140-84.us-west-2.compute.internal 45m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l to ip-10-0-142-127.us-west-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws to ip-10-0-142-127.us-west-2.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-t9zcq to ip-10-0-142-127.us-west-2.compute.internal 32m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-qr6gj to ip-10-0-129-180.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-r5grr to ip-10-0-129-180.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Successfully assigned openshift-ingress/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w to ip-10-0-140-84.us-west-2.compute.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-s2qtj to ip-10-0-140-84.us-west-2.compute.internal 60m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-4lhhw to ip-10-0-129-180.us-west-2.compute.internal 33m Normal Scheduled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Successfully assigned openshift-ingress/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv to ip-10-0-142-127.us-west-2.compute.internal 48m Normal Scheduled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Successfully assigned openshift-ingress/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 to ip-10-0-140-84.us-west-2.compute.internal 60m Normal Scheduled pod/payload-pre-processing-7b98bb678c-7bknp Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-7bknp to ip-10-0-129-180.us-west-2.compute.internal 21m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-s8bll to ip-10-0-140-84.us-west-2.compute.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-fcb75c45f-sjzsr to ip-10-0-140-84.us-west-2.compute.internal 76m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-746fc4dd59 from 0 to 1 76m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 76m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 76m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 75m Normal SuccessfulCreate replicaset/router-default-746fc4dd59 Created pod: router-default-746fc4dd59-qfxr9 70m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 69m Warning FailedMount pod/router-default-746fc4dd59-qfxr9 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 69m Warning FailedMount pod/router-default-746fc4dd59-qfxr9 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 69m Warning FailedMount pod/router-default-746fc4dd59-qfxr9 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 69m Warning FailedMount pod/router-default-746fc4dd59-qfxr9 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 69m Normal Pulled pod/router-default-746fc4dd59-qfxr9 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 69m Normal Started pod/router-default-746fc4dd59-qfxr9 Container started 69m Normal Created pod/router-default-746fc4dd59-qfxr9 Container created 69m Normal AddedInterface pod/router-default-746fc4dd59-qfxr9 Add eth0 [10.132.0.9/23] from ovn-kubernetes 66m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 66m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-mhsg7 66m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 66m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Add eth0 [10.133.0.18/23] from ovn-kubernetes 66m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 66m Normal Created pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Container created 66m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.032s (4.032s including waiting). Image size: 185476477 bytes. 66m Normal Started pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Container started 66m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Readiness probe failed: HTTP probe failed with statuscode: 503 66m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Readiness probe error: HTTP probe failed with statuscode: 503... 66m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-fcb75c45f from 0 to 1 66m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-74987c9b47 Created pod: openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl 66m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-74987c9b47 from 0 to 1 66m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-jdhp6 66m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 66m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Add eth0 [10.132.0.34/23] from ovn-kubernetes 66m Normal Pulling pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 66m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl Add eth0 [10.132.0.35/23] from ovn-kubernetes 65m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 65m Normal Started pod/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl Container started 65m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.885s (13.885s including waiting). Image size: 183555333 bytes. 65m Normal Created pod/openshift-ai-inference-openshift-ai-inference-74987c9b47-s4mjl Container created 65m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.938s (13.938s including waiting). Image size: 183555333 bytes. 65m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 65m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 65m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 64m Normal SuccessfulCreate replicaset/kube-auth-proxy-744f97f697 Created pod: kube-auth-proxy-744f97f697-pfr86 64m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-7c689579c from 0 to 1 64m Normal AddedInterface pod/kube-auth-proxy-744f97f697-vjlc2 Add eth0 [10.134.0.24/23] from ovn-kubernetes 64m Normal Pulling pod/kube-auth-proxy-744f97f697-vjlc2 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 64m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-744f97f697 from 0 to 2 64m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-7c689579c Created pod: data-science-gateway-data-science-gateway-class-7c689579c-cjljn 64m Normal SuccessfulCreate replicaset/kube-auth-proxy-744f97f697 Created pod: kube-auth-proxy-744f97f697-vjlc2 64m Normal Started pod/data-science-gateway-data-science-gateway-class-7c689579c-cjljn Container started 64m Normal Created pod/data-science-gateway-data-science-gateway-class-7c689579c-cjljn Container created 64m Normal Pulled pod/data-science-gateway-data-science-gateway-class-7c689579c-cjljn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 64m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-7c689579c-cjljn Add eth0 [10.132.0.48/23] from ovn-kubernetes 64m Normal AddedInterface pod/kube-auth-proxy-744f97f697-pfr86 Add eth0 [10.133.0.20/23] from ovn-kubernetes 64m Normal Pulling pod/kube-auth-proxy-744f97f697-pfr86 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 64m Normal Created pod/kube-auth-proxy-744f97f697-vjlc2 Container created 64m Normal Created pod/kube-auth-proxy-744f97f697-pfr86 Container created 64m Normal Started pod/kube-auth-proxy-744f97f697-pfr86 Container started 64m Normal Started pod/kube-auth-proxy-744f97f697-vjlc2 Container started 64m Normal Pulled pod/kube-auth-proxy-744f97f697-pfr86 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.899s (2.899s including waiting). Image size: 188904677 bytes. 64m Normal Pulled pod/kube-auth-proxy-744f97f697-vjlc2 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.329s (3.329s including waiting). Image size: 188904677 bytes. 64m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 64m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 63m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 63m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 63m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Add eth0 [10.133.0.23/23] from ovn-kubernetes 63m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Container created 63m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Container started 63m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-ndttp 63m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Readiness probe failed: HTTP probe failed with statuscode: 503 63m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Readiness probe error: HTTP probe failed with statuscode: 503... 63m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Readiness probe failed: Get "http://10.133.0.18:8080/ready": dial tcp 10.133.0.18:8080: connect: connection refused 63m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Readiness probe error: Get "http://10.133.0.18:8080/ready": dial tcp 10.133.0.18:8080: connect: connection refused... 63m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 63m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-mhsg7 Stopping container discovery 63m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-mhsg7 62m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Readiness probe error: Get "http://10.132.0.34:15021/healthz/ready": read tcp 10.132.0.2:54988->10.132.0.34:15021: read: connection reset by peer... 62m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Readiness probe failed: Get "http://10.132.0.34:15021/healthz/ready": read tcp 10.132.0.2:54988->10.132.0.34:15021: read: connection reset by peer 62m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Container created 62m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Container started 62m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused... 62m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=app%3Distiod%2Cistio.io%2Frev%3Dopenshift-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=app%3Distiod%2Cistio.io%2Frev%3Dopenshift-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Ddata-science-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Ddata-science-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Dmaas-default-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Dmaas-default-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=app%3Distiod%2Cistio.io%2Frev%3Dopenshift-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=app%3Distiod%2Cistio.io%2Frev%3Dopenshift-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=app%3Dkube-auth-proxy\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Ddata-science-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Dmaas-default-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Dmaas-default-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Dopenshift-ai-inference\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Dopenshift-ai-inference\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=app%3Dkube-auth-proxy\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: an error on the server ("Internal Server Error: \"/apis/metrics.k8s.io/v1beta1/namespaces/openshift-ingress/pods?labelSelector=gateway.networking.k8s.io%2Fgateway-name%3Ddata-science-gateway\": Post \"https://172.31.0.1:443/apis/authorization.k8s.io/v1/subjectaccessreviews?timeout=10s\": context deadline exceeded") has prevented the request from succeeding (get pods.metrics.k8s.io) 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-4lhhw 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-7c9qm 60m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-fcb75c45f from 1 to 4 60m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 60m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-cnft9 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Add eth0 [10.133.0.31/23] from ovn-kubernetes 60m Normal Pulling pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 60m Normal Pulling pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Add eth0 [10.134.0.36/23] from ovn-kubernetes 60m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Add eth0 [10.133.0.30/23] from ovn-kubernetes 60m Normal Pulling pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 60m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Container started 60m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Container created 60m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.205s (4.205s including waiting). Image size: 183555333 bytes. 60m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Container created 60m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.377s (4.377s including waiting). Image size: 183555333 bytes. 60m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Container started 60m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.37s (4.37s including waiting). Image size: 183555333 bytes. 60m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Container started 60m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Container created 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Startup probe failed: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Startup probe error: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused... 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Startup probe failed: Get "http://10.133.0.30:15021/healthz/ready": dial tcp 10.133.0.30:15021: connect: connection refused 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Startup probe error: Get "http://10.133.0.31:15021/healthz/ready": dial tcp 10.133.0.31:15021: connect: connection refused... 60m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused... 60m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 60m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 60m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-9f4p2 60m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-7bknp 60m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 60m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 60m Normal Pulling pod/payload-pre-processing-7b98bb678c-7bknp Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal Pulling pod/payload-processing-55fbbbf9dc-9f4p2 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 60m Normal AddedInterface pod/payload-processing-55fbbbf9dc-9f4p2 Add eth0 [10.133.0.33/23] from ovn-kubernetes 60m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-7bknp Add eth0 [10.134.0.37/23] from ovn-kubernetes 60m Normal Pulled pod/payload-processing-55fbbbf9dc-9f4p2 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.037s (3.037s including waiting). Image size: 125353963 bytes. 60m Normal Created pod/payload-pre-processing-7b98bb678c-7bknp Container created 60m Normal Pulled pod/payload-pre-processing-7b98bb678c-7bknp Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.945s (2.945s including waiting). Image size: 125353963 bytes. 60m Normal Started pod/payload-pre-processing-7b98bb678c-7bknp Container started 60m Normal Created pod/payload-processing-55fbbbf9dc-9f4p2 Container created 60m Normal Started pod/payload-processing-55fbbbf9dc-9f4p2 Container started 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-fcb75c45f from 4 to 5 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-d7kw8 59m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 59m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Container started 59m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Container created 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Add eth0 [10.134.0.39/23] from ovn-kubernetes 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-d7kw8 Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 59m Normal AddedLabel gateway/e2e-worker-w1-4fe3a4 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-4fe3a4 59m Normal AddedLabel gateway/e2e-worker-w5-ef6d93 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-ef6d93 59m Normal AddedLabel gateway/e2e-worker-w2-e7f13a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-e7f13a 59m Normal AddedLabel gateway/e2e-worker-w0-ac1273 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-ac1273 59m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled up replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv 59m Normal AddedLabel gateway/e2e-worker-w3-f7821a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-f7821a 59m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk 59m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled up replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 0 to 1 59m Normal AddedLabel gateway/e2e-worker-w4-59357a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-59357a 59m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled up replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r 59m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Add eth0 [10.133.0.34/23] from ovn-kubernetes 59m Normal AddedLabel gateway/e2e-worker-w6-a1685e Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-a1685e 59m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled up replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Created pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk 59m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Add eth0 [10.133.0.36/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Add eth0 [10.133.0.35/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled up replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g 59m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Add eth0 [10.134.0.40/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled up replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 0 to 1 59m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x 59m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Add eth0 [10.134.0.42/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Add eth0 [10.134.0.41/23] from ovn-kubernetes 59m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Add eth0 [10.133.0.37/23] from ovn-kubernetes 59m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx 59m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled up replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f Created pod: payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-4fe3a4 Scaled up replica set payload-processing-e2e-worker-w1-4fe3a4-6474577947 from 0 to 1 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-4fe3a4-6474577947 Error creating: pods "payload-processing-e2e-worker-w1-4fe3a4-6474577947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-4fe3a4: serviceaccount "payload-processing-e2e-worker-w1-4fe3a4" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-4fe3a4-6474577947 Created pod: payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f Error creating: pods "payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-4fe3a4: serviceaccount "payload-processing-e2e-worker-w1-4fe3a4" not found 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-4fe3a4 Scaled up replica set payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f from 0 to 1 59m Normal AddedInterface pod/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh Add eth0 [10.134.0.44/23] from ovn-kubernetes 59m Normal Started pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Container started 59m Normal Started pod/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh Container started 59m Normal Created pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Container created 59m Normal Pulled pod/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh Container created 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Add eth0 [10.134.0.43/23] from ovn-kubernetes 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-a1685e Scaled up replica set payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747 from 0 to 1 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-a1685e Scaled up replica set payload-processing-e2e-worker-w6-a1685e-7f8694fc87 from 0 to 1 59m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Add eth0 [10.133.0.38/23] from ovn-kubernetes 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-a1685e-7f8694fc87 Error creating: pods "payload-processing-e2e-worker-w6-a1685e-7f8694fc87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-a1685e: serviceaccount "payload-processing-e2e-worker-w6-a1685e" not found 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747 Error creating: pods "payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-a1685e: serviceaccount "payload-processing-e2e-worker-w6-a1685e" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-a1685e-7f8694fc87 Created pod: payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747 Created pod: payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv 59m Normal Started pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Container started 59m Normal Started pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Container started 59m Normal Pulled pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Normal Created pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Container created 59m Normal Created pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Container created 59m Normal AddedInterface pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Add eth0 [10.133.0.39/23] from ovn-kubernetes 59m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-f7821a Scaled up replica set payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899 from 0 to 1 59m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899 Error creating: pods "payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-f7821a: serviceaccount "payload-processing-e2e-worker-w3-f7821a" not found 59m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755 Error creating: pods "payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-f7821a: serviceaccount "payload-processing-e2e-worker-w3-f7821a" not found 59m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755 Created pod: payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 59m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-f7821a Scaled up replica set payload-processing-e2e-worker-w3-f7821a-7bd5cd9755 from 0 to 1 59m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899 Created pod: payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb 58m Normal AddedInterface pod/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 Add eth0 [10.134.0.45/23] from ovn-kubernetes 58m Normal Created pod/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb Container created 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb Add eth0 [10.133.0.40/23] from ovn-kubernetes 58m Normal Started pod/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 Container started 58m Normal Created pod/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 Container created 58m Normal Pulled pod/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Started pod/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb Container started 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-ac1273 Scaled up replica set payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5 from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-ac1273-74fc6446bf Error creating: pods "payload-processing-e2e-worker-w0-ac1273-74fc6446bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ac1273: serviceaccount "payload-processing-e2e-worker-w0-ac1273" not found 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5 Error creating: pods "payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-ac1273: serviceaccount "payload-processing-e2e-worker-w0-ac1273" not found 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-ac1273 Scaled up replica set payload-processing-e2e-worker-w0-ac1273-74fc6446bf from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5 Created pod: payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q 58m Normal Started pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Container started 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-ac1273-74fc6446bf Created pod: payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 58m Normal Created pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Container created 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Add eth0 [10.134.0.46/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Add eth0 [10.134.0.47/23] from ovn-kubernetes 58m Normal Pulled pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Started pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Container started 58m Normal Created pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Container created 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45 Error creating: pods "payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-e7f13a: serviceaccount "payload-processing-e2e-worker-w2-e7f13a" not found 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45 Created pod: payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-e7f13a Scaled up replica set payload-processing-e2e-worker-w2-e7f13a-687d7c9f45 from 0 to 1 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787 Error creating: pods "payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-e7f13a: serviceaccount "payload-processing-e2e-worker-w2-e7f13a" not found 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787 Created pod: payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-e7f13a Scaled up replica set payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787 from 0 to 1 58m Normal Started pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Container started 58m Normal Pulled pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Container created 58m Normal AddedInterface pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Add eth0 [10.134.0.49/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Add eth0 [10.134.0.48/23] from ovn-kubernetes 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Container created 58m Normal Started pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Container started 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-59357a Scaled up replica set payload-processing-e2e-worker-w4-59357a-7df684b4b7 from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-59357a-7df684b4b7 Error creating: pods "payload-processing-e2e-worker-w4-59357a-7df684b4b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-59357a: serviceaccount "payload-processing-e2e-worker-w4-59357a" not found 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975 Error creating: pods "payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-59357a: serviceaccount "payload-processing-e2e-worker-w4-59357a" not found 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-59357a Scaled up replica set payload-pre-processing-e2e-worker-w4-59357a-88f6c5975 from 0 to 1 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Add eth0 [10.133.0.42/23] from ovn-kubernetes 58m Normal Started pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Container started 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975 Created pod: payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-59357a-7df684b4b7 Created pod: payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls 58m Normal Created pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Container created 58m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-ef6d93 Scaled up replica set payload-processing-e2e-worker-w5-ef6d93-847d6594c9 from 0 to 1 58m Normal AddedInterface pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Add eth0 [10.133.0.43/23] from ovn-kubernetes 58m Normal Started pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Container started 58m Normal Pulled pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Container created 58m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Container created 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499 Error creating: pods "payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-ef6d93: serviceaccount "payload-processing-e2e-worker-w5-ef6d93" not found 58m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499 Created pod: payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 58m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Container started 58m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-ef6d93 Scaled up replica set payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499 from 0 to 1 58m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-ef6d93-847d6594c9 Error creating: pods "payload-processing-e2e-worker-w5-ef6d93-847d6594c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-ef6d93: serviceaccount "payload-processing-e2e-worker-w5-ef6d93" not found 58m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-ef6d93-847d6594c9 Created pod: payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n 58m Normal Started pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Container started 58m Normal Created pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Container created 58m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Add eth0 [10.133.0.44/23] from ovn-kubernetes 58m Normal Pulled pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Created pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Container created 58m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 58m Normal Started pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Container started 58m Normal AddedInterface pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Add eth0 [10.134.0.50/23] from ovn-kubernetes 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 58m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Container created 58m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Container started 58m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Container started 58m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Container created 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Startup probe failed: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Startup probe error: Get "http://10.133.0.35:15021/healthz/ready": dial tcp 10.133.0.35:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled up replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 1 to 4 58m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 58m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 58m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Add eth0 [10.132.0.65/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Container started 58m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Container started 58m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Container created 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Add eth0 [10.133.0.46/23] from ovn-kubernetes 58m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Container started 58m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Container created 58m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Add eth0 [10.134.0.51/23] from ovn-kubernetes 58m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Container created 58m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Container created 58m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Container started 58m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 58m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused... 58m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Container created 58m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Container started 58m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Startup probe failed: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Startup probe error: Get "http://10.133.0.34:15021/healthz/ready": dial tcp 10.133.0.34:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": read tcp 10.134.0.2:42014->10.134.0.41:15021: read: connection reset by peer 58m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": read tcp 10.134.0.2:42014->10.134.0.41:15021: read: connection reset by peer... 58m Normal Created pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Container created 58m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": read tcp 10.134.0.2:58434->10.134.0.42:15021: read: connection reset by peer 58m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Container started 58m Normal Pulled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Started pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Container started 58m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Container created 58m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": read tcp 10.134.0.2:58434->10.134.0.42:15021: read: connection reset by peer... 58m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 58m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 58m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 58m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled up replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 4 to 5 58m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt 57m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Add eth0 [10.134.0.52/23] from ovn-kubernetes 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Container started 57m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Container created 57m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Container started 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled up replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 1 to 2 55m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Add eth0 [10.133.0.52/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Container created 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w 55m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 54m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-7c9qm Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-4lhhw 54m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-4lhhw Stopping container istio-proxy 54m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-7c9qm 53m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Add eth0 [10.134.0.79/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled up replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 1 to 2 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 53m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Container created 53m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Container started 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-jbh6n 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-zvlv7 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled up replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 1 to 2 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-dgfgj 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Container created 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Container started 53m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Add eth0 [10.134.0.80/23] from ovn-kubernetes 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Container created 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Add eth0 [10.134.0.81/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Container started 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Container started 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Container started 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Add eth0 [10.133.0.56/23] from ovn-kubernetes 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Add eth0 [10.132.0.73/23] from ovn-kubernetes 53m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 53m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Add eth0 [10.133.0.58/23] from ovn-kubernetes 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled up replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 2 to 4 53m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-8569j 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx 53m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Add eth0 [10.132.0.74/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf 53m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled up replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 5 to 7 53m Normal Started pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Container started 53m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Container started 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled up replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 1 to 2 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f Created pod: maas-default-gateway-openshift-default-fcb75c45f-s2qtj 53m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 53m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Created pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd 53m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Add eth0 [10.132.0.75/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Add eth0 [10.133.0.59/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Container started 53m Normal AddedInterface pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Add eth0 [10.133.0.60/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Container created 53m Normal Created pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Container created 53m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Container started 53m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Add eth0 [10.133.0.62/23] from ovn-kubernetes 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Add eth0 [10.132.0.76/23] from ovn-kubernetes 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Container created 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Add eth0 [10.133.0.61/23] from ovn-kubernetes 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Container started 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Container started 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Container created 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Add eth0 [10.134.0.85/23] from ovn-kubernetes 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Container created 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Container started 53m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Container started 53m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Container created 53m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Startup probe failed: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 53m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 53m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Startup probe error: Get "http://10.133.0.61:15021/healthz/ready": dial tcp 10.133.0.61:15021: connect: connection refused... 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled up replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 1 to 2 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Container started 53m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Container created 53m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Add eth0 [10.134.0.87/23] from ovn-kubernetes 53m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Startup probe failed: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Startup probe error: Get "http://10.134.0.87:15021/healthz/ready": dial tcp 10.134.0.87:15021: connect: connection refused... 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled up replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 2 to 3 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled up replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 1 to 2 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 53m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Add eth0 [10.132.0.78/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Container started 53m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Add eth0 [10.133.0.63/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Container created 53m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Container created 53m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Container started 53m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Startup probe failed: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Startup probe error: Get "http://10.133.0.63:15021/healthz/ready": dial tcp 10.133.0.63:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp 52m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Add eth0 [10.134.0.88/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled up replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 7 to 8 52m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Container created 52m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Container started 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 52m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled up replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 8 to 10 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled up replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 2 to 3 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Created pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 (combined from similar events): Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Created pod: e2e-worker-w0-ac1273-openshift-default-79699df87-stllq 52m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Add eth0 [10.132.0.81/23] from ovn-kubernetes 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Container created 52m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Container created 52m Normal Pulled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Container started 52m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Container started 52m Normal Started pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Container started 52m Normal AddedInterface pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Add eth0 [10.132.0.82/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Container created 52m Normal Pulled pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Add eth0 [10.133.0.65/23] from ovn-kubernetes 52m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Startup probe error: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Startup probe failed: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Startup probe error: Get "http://10.133.0.65:15021/healthz/ready": dial tcp 10.133.0.65:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Startup probe failed: Get "http://10.132.0.81:15021/healthz/ready": dial tcp 10.132.0.81:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled up replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 4 to 7 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled up replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 2 to 3 52m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Add eth0 [10.133.0.67/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Container started 52m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Container created 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl 52m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Add eth0 [10.132.0.83/23] from ovn-kubernetes 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled up replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 2 to 4 52m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Container created 52m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Add eth0 [10.133.0.66/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Add eth0 [10.134.0.91/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Add eth0 [10.132.0.84/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Container created 52m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Container started 52m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Container created 52m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Container created 52m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Container started 52m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Container started 52m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Add eth0 [10.132.0.85/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Container started 52m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Container started 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 52m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Container created 52m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Startup probe failed: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Startup probe error: Get "http://10.132.0.83:15021/healthz/ready": dial tcp 10.132.0.83:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Startup probe failed: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Startup probe error: Get "http://10.133.0.66:15021/healthz/ready": dial tcp 10.133.0.66:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled up replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 3 to 5 51m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Container created 51m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Container started 51m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Container started 51m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Add eth0 [10.134.0.93/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Container created 51m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Add eth0 [10.133.0.68/23] from ovn-kubernetes 51m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled up replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 2 to 4 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk 51m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Container created 51m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Add eth0 [10.132.0.86/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Add eth0 [10.133.0.69/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc 51m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Container started 51m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Container created 51m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Container started 51m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Startup probe error: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Startup probe failed: Get "http://10.133.0.69:15021/healthz/ready": dial tcp 10.133.0.69:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Created pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 50m Normal AddedInterface pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Add eth0 [10.132.0.87/23] from ovn-kubernetes 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Created pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Created pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh 50m Normal AddedInterface pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Add eth0 [10.133.0.71/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled up replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 3 to 6 50m Normal AddedInterface pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Add eth0 [10.134.0.98/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Container started 50m Normal Created pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Container created 50m Normal Pulled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Container created 50m Normal Pulled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Container started 50m Normal Created pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Container created 50m Normal Pulled pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Container started 50m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled up replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 4 to 7 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k 50m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Add eth0 [10.132.0.89/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Container started 50m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Container created 50m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Container started 50m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Container created 50m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Add eth0 [10.134.0.102/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Add eth0 [10.134.0.101/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Container created 50m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Container started 50m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Readiness probe failed: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Readiness probe error: Get "http://10.134.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Readiness probe error: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Readiness probe failed: Get "http://10.132.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled up replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 4 to 8 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled up replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 3 to 6 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Container started 49m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Container started 49m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Add eth0 [10.132.0.90/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Container created 49m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Container started 49m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Add eth0 [10.132.0.91/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s 49m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Container created 49m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Add eth0 [10.133.0.74/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Add eth0 [10.133.0.75/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Container started 49m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Container created 49m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Container created 49m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Add eth0 [10.134.0.106/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Container started 49m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Container created 49m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Container started 49m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled up replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 7 to 10 49m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Add eth0 [10.134.0.105/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f (combined from similar events): Created pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv 49m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Add eth0 [10.134.0.107/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Container created 49m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Container started 49m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Container created 49m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Add eth0 [10.132.0.92/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Add eth0 [10.134.0.108/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Container created 49m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Container started 49m Normal AddedInterface pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Add eth0 [10.133.0.76/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Container created 49m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Container started 49m Normal Created pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Container created 49m Normal Started pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Container started 49m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Startup probe failed: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Startup probe error: Get "http://10.134.0.105:15021/healthz/ready": dial tcp 10.134.0.105:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": dial tcp 10.134.0.106:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": dial tcp 10.134.0.107:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Startup probe failed: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Startup probe error: Get "http://10.134.0.107:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Startup probe failed: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Startup probe error: Get "http://10.134.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Startup probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Startup probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled up replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 8 to 10 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 (combined from similar events): Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled up replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 6 to 8 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Created pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 10; reason: All metrics below target 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 8; reason: 49m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Container created 49m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Container started 49m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Add eth0 [10.134.0.112/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Add eth0 [10.133.0.78/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Container started 49m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Container created 49m Normal AddedInterface pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Add eth0 [10.132.0.93/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Add eth0 [10.133.0.77/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Container created 49m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Container started 49m Normal Pulled pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Container started 49m Normal Created pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Container created 49m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c (combined from similar events): Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 49m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled up replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 7 to 10 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled up replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 5 to 10 49m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Container started 49m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Container created 49m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Add eth0 [10.132.0.94/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Container created 49m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Container started 49m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Container created 49m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Add eth0 [10.133.0.79/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Add eth0 [10.134.0.114/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 (combined from similar events): Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Created pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt 49m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Container started 49m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Container started 49m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Container created 49m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Add eth0 [10.133.0.81/23] from ovn-kubernetes 49m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Container created 49m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Add eth0 [10.134.0.115/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Container started 49m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Container created 49m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Add eth0 [10.132.0.95/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Container started 49m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Container created 49m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Add eth0 [10.133.0.80/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Container started 49m Normal Created pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Container created 49m Normal Pulled pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Add eth0 [10.132.0.96/23] from ovn-kubernetes 49m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Readiness probe error: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Readiness probe failed: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Readiness probe error: Get "http://10.134.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Readiness probe failed: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Readiness probe error: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Readiness probe failed: Get "http://10.133.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Readiness probe error: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Readiness probe failed: Get "http://10.133.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Startup probe failed: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Startup probe error: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-fcb75c45f from 10 to 7 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-jbh6n 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-s2qtj 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-s2qtj Stopping container istio-proxy 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-jbh6n Stopping container istio-proxy 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-h8jrx Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-h8jrx 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Add eth0 [10.134.0.126/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-9d8tx 48m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 48m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.392s (4.392s including waiting). Image size: 185476477 bytes. 48m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Container created 48m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Container started 48m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-9d8tx Readiness probe failed: HTTP probe failed with statuscode: 503 48m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Readiness probe failed: Get "http://10.133.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Readiness probe error: Get "http://10.133.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Readiness probe failed: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Readiness probe error: Get "http://10.134.0.108:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 (combined from similar events): Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Created pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled up replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 8 to 10 48m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Container started 48m Normal Started pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Container started 48m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Add eth0 [10.133.0.83/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Container created 48m Normal AddedInterface pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Add eth0 [10.132.0.98/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Container created 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-dgfgj 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-fqh2n Stopping container istio-proxy 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-dgfgj Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-fcb75c45f from 7 to 4 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-fqh2n 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-vkxlw Stopping container istio-proxy 48m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-vkxlw 48m Normal AddedLabel gateway/e2e-migrate-f27146d0-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-f27146d0-gw 48m Normal SuccessfulCreate replicaset/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f Created pod: e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb 48m Normal ScalingReplicaSet deployment/e2e-migrate-f27146d0-gw-openshift-default Scaled up replica set e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f from 0 to 1 48m Normal AddedInterface pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Add eth0 [10.133.0.84/23] from ovn-kubernetes 48m Normal AddedLabel gateway/e2e-ait-a08562e5 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a08562e5 48m Normal ScalingReplicaSet deployment/e2e-ait-a08562e5-openshift-default Scaled up replica set e2e-ait-a08562e5-openshift-default-c86877577 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-ait-a08562e5-openshift-default-c86877577 Created pod: e2e-ait-a08562e5-openshift-default-c86877577-gv4tf 48m Normal AddedInterface pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Add eth0 [10.133.0.85/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 48m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 48m Normal AddedLabel gateway/e2e-derive-caa664e1 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-caa664e1 48m Normal ScalingReplicaSet deployment/e2e-derive-caa664e1-openshift-default Scaled up replica set e2e-derive-caa664e1-openshift-default-bd4767cd9 from 0 to 1 48m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Add eth0 [10.133.0.86/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9 Created pod: payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-f27146d0 Scaled up replica set payload-processing-e2e-migrate-f27146d0-775cb499fc from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9 Error creating: pods "payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-f27146d0: serviceaccount "payload-processing-e2e-migrate-f27146d0" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-f27146d0 Scaled up replica set payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-migrate-f27146d0-775cb499fc Error creating: pods "payload-processing-e2e-migrate-f27146d0-775cb499fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-f27146d0: serviceaccount "payload-processing-e2e-migrate-f27146d0" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-f27146d0-775cb499fc Created pod: payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk 48m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp Add eth0 [10.133.0.87/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp Container started 48m Normal Created pod/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp Container created 48m Normal Pulled pod/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk Add eth0 [10.133.0.88/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk Container created 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-a08562e5 Scaled up replica set payload-processing-e2e-ait-a08562e5-57698b7cdc from 0 to 1 48m Normal Started pod/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk Container started 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-a08562e5-57698b7cdc Created pod: payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-a08562e5-57698b7cdc Error creating: pods "payload-processing-e2e-ait-a08562e5-57698b7cdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a08562e5: serviceaccount "payload-processing-e2e-ait-a08562e5" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659 Error creating: pods "payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a08562e5: serviceaccount "payload-processing-e2e-ait-a08562e5" not found 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Add eth0 [10.132.0.99/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659 Created pod: payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-a08562e5 Scaled up replica set payload-pre-processing-e2e-ait-a08562e5-c6f5dd659 from 0 to 1 48m Normal Pulling pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 48m Normal Created pod/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 Container created 48m Normal AddedInterface pod/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 Add eth0 [10.133.0.89/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 Container started 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-fcb75c45f from 4 to 3 48m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-zvlv7 Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-zvlv7 48m Normal Pulled pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.851s (2.851s including waiting). Image size: 125353963 bytes. 48m Normal Created pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Container created 48m Normal Started pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Container started 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-caa664e1 Scaled up replica set payload-pre-processing-e2e-derive-caa664e1-979589fc5 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-derive-caa664e1-857c657ccc Error creating: pods "payload-processing-e2e-derive-caa664e1-857c657ccc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-caa664e1: serviceaccount "payload-processing-e2e-derive-caa664e1" not found 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-caa664e1-979589fc5 Error creating: pods "payload-pre-processing-e2e-derive-caa664e1-979589fc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-caa664e1: serviceaccount "payload-processing-e2e-derive-caa664e1" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-caa664e1 Scaled up replica set payload-processing-e2e-derive-caa664e1-857c657ccc from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-caa664e1-857c657ccc Created pod: payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-caa664e1-979589fc5 Created pod: payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh 48m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Readiness probe failed: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Readiness probe failed: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Readiness probe error: Get "http://10.133.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Readiness probe failed: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Readiness probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Readiness probe error: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Readiness probe error: Get "http://10.134.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Readiness probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Readiness probe error: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Readiness probe failed: Get "http://10.132.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Readiness probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Readiness probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Readiness probe failed: timeout: failed to connect service "10.134.0.50:50052" within 1s: context deadline exceeded 48m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Readiness probe error: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Readiness probe failed: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Readiness probe error: Get "http://10.134.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Readiness probe failed: Get "http://10.133.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal AddedInterface pod/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x Add eth0 [10.133.0.91/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Readiness probe failed: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Readiness probe error: Get "http://10.134.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Readiness probe failed: Get "http://10.134.0.114:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Readiness probe error: Get "http://10.133.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal AddedInterface pod/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh Add eth0 [10.133.0.90/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Pulled pod/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x Container created 47m Normal Started pod/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x Container started 47m Normal Started pod/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh Container started 47m Normal Created pod/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh Container created 47m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Add eth0 [10.134.0.139/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Add eth0 [10.132.0.103/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Add eth0 [10.133.0.92/23] from ovn-kubernetes 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Container created 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Container started 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Container created 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Container started 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Container started 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Container created 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Startup probe error: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Startup probe failed: Get "http://10.134.0.139:15021/healthz/ready": dial tcp 10.134.0.139:15021: connect: connection refused 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Add eth0 [10.134.0.140/23] from ovn-kubernetes 46m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: All metrics below target 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Add eth0 [10.132.0.104/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-fcb75c45f (combined from similar events): Created pod: maas-default-gateway-openshift-default-fcb75c45f-mb94r 46m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-fcb75c45f from 6 to 10 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Add eth0 [10.134.0.141/23] from ovn-kubernetes 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Container started 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Container started 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Container created 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Container started 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Container created 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Add eth0 [10.133.0.93/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Container created 46m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Container created 46m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Container started 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 46m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 46m Normal ScalingReplicaSet deployment/e2e-derive-caa664e1-openshift-default Scaled up replica set e2e-derive-caa664e1-openshift-default-bd4767cd9 from 1 to 4 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 46m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-derive-caa664e1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr 46m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Container started 46m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Container created 46m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Container started 46m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Add eth0 [10.134.0.142/23] from ovn-kubernetes 46m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Container started 46m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Add eth0 [10.133.0.94/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Container created 46m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Add eth0 [10.132.0.105/23] from ovn-kubernetes 46m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Container created 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 46m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Readiness probe error: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Readiness probe failed: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 46m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Startup probe failed: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused 46m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Startup probe error: Get "http://10.132.0.105:15021/healthz/ready": dial tcp 10.132.0.105:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 46m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 46m Normal Pulled pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Container started 46m Normal Created pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Container created 46m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Container created 46m Normal Started pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Container started 46m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Container started 46m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Container created 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-f27146d0-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-f27146d0-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a08562e5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a08562e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/payload-processing-e2e-migrate-f27146d0-775cb499fc-8qhmk Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-migrate-f27146d0-5976fb74f9-vxqsp Stopping container payload-pre-processing 46m Warning ProbeError pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 46m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-caa664e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-caa664e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 46m Normal Killing pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-ait-a08562e5-57698b7cdc-xj429 Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-derive-caa664e1-979589fc5-jckbh Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-derive-caa664e1-857c657ccc-77m9x Stopping container payload-processing 46m Warning Unhealthy pod/payload-pre-processing-e2e-ait-a08562e5-c6f5dd659-nd2ws Readiness probe failed: timeout: failed to connect service "10.132.0.99:50052" within 1s: context deadline exceeded 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-caa664e1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-caa664e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-f27146d0-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-f27146d0-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a08562e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a08562e5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l 45m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc 45m Normal SuccessfulCreate replicaset/e2e-derive-caa664e1-openshift-default-bd4767cd9 Created pod: e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf 45m Normal ScalingReplicaSet deployment/e2e-derive-caa664e1-openshift-default Scaled up replica set e2e-derive-caa664e1-openshift-default-bd4767cd9 from 4 to 7 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-derive-caa664e1-openshift-default New size: 7; reason: All metrics below target 45m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Add eth0 [10.132.0.106/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Add eth0 [10.133.0.95/23] from ovn-kubernetes 45m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Container created 45m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Container started 45m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Container created 45m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Container started 45m Normal Started pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Container started 45m Normal Pulled pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Container created 45m Normal AddedInterface pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Add eth0 [10.134.0.147/23] from ovn-kubernetes 45m Normal Killing pod/e2e-migrate-f27146d0-gw-openshift-default-5b4f7cdf6f-c8vdb Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Startup probe failed: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Startup probe failed: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused 45m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Startup probe error: Get "http://10.134.0.147:15021/healthz/ready": dial tcp 10.134.0.147:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Startup probe error: Get "http://10.132.0.106:15021/healthz/ready": dial tcp 10.132.0.106:15021: connect: connection refused... 45m Normal AddedLabel gateway/e2e-ait-c1bb9544 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c1bb9544 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-f4vwk Stopping container istio-proxy 45m Normal AddedLabel gateway/e2e-ait-ce4664d0 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ce4664d0 45m Normal AddedInterface pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Add eth0 [10.132.0.107/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-ait-c1bb9544-openshift-default Scaled up replica set e2e-ait-c1bb9544-openshift-default-6467754d77 from 0 to 1 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-npbdr Stopping container istio-proxy 45m Normal Killing pod/e2e-ait-a08562e5-openshift-default-c86877577-gv4tf Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-ait-c1bb9544-openshift-default-6467754d77 Created pod: e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-gctb6 Stopping container istio-proxy 45m Warning ProbeError pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 45m Warning FailedToRetrieveImagePullSecret pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Unable to retrieve some image pull secrets (e2e-derive-caa664e1-openshift-default-dockercfg-cbxhc); attempting to pull the image may not succeed. 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-ngbsf Stopping container istio-proxy 45m Warning FailedToRetrieveImagePullSecret pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Unable to retrieve some image pull secrets (e2e-derive-caa664e1-openshift-default-dockercfg-cbxhc); attempting to pull the image may not succeed. 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-vcnp4 Stopping container istio-proxy 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-tq46l Stopping container istio-proxy 45m Normal Killing pod/e2e-derive-caa664e1-openshift-default-bd4767cd9-pwchc Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-ait-ce4664d0-openshift-default-5c55fff65c Created pod: e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw 45m Normal ScalingReplicaSet deployment/e2e-ait-ce4664d0-openshift-default Scaled up replica set e2e-ait-ce4664d0-openshift-default-5c55fff65c from 0 to 1 45m Normal AddedInterface pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Add eth0 [10.133.0.96/23] from ovn-kubernetes 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c Error creating: pods "payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c1bb9544: serviceaccount "payload-processing-e2e-ait-c1bb9544" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-c1bb9544 Scaled up replica set payload-processing-e2e-ait-c1bb9544-7567c84675 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-c1bb9544 Scaled up replica set payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c Created pod: payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-c1bb9544-7567c84675 Created pod: payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-c1bb9544-7567c84675 Error creating: pods "payload-processing-e2e-ait-c1bb9544-7567c84675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c1bb9544: serviceaccount "payload-processing-e2e-ait-c1bb9544" not found 45m Normal Created pod/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 Container created 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 Add eth0 [10.133.0.97/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj Add eth0 [10.133.0.98/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 Container started 45m Normal Pulled pod/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj Container created 45m Normal Started pod/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj Container started 45m Warning FailedCreate replicaset/payload-processing-e2e-ait-ce4664d0-56c8dc8f55 Error creating: pods "payload-processing-e2e-ait-ce4664d0-56c8dc8f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ce4664d0: serviceaccount "payload-processing-e2e-ait-ce4664d0" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-ce4664d0 Scaled up replica set payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f Error creating: pods "payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ce4664d0: serviceaccount "payload-processing-e2e-ait-ce4664d0" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f Created pod: payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-ce4664d0 Scaled up replica set payload-processing-e2e-ait-ce4664d0-56c8dc8f55 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-ce4664d0-56c8dc8f55 Created pod: payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh 45m Normal AddedInterface pod/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q Add eth0 [10.133.0.99/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q Container created 45m Normal Started pod/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q Container started 45m Normal Started pod/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh Container started 45m Normal Created pod/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh Container created 45m Normal Pulled pod/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh Add eth0 [10.133.0.100/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled down replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 6 to 5 45m Normal Killing pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Deleted pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-2lprn 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 5; reason: All metrics below target 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ce4664d0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ce4664d0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Readiness probe failed: Get "http://10.134.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Readiness probe error: Get "http://10.132.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Readiness probe failed: Get "http://10.132.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Readiness probe error: Get "http://10.134.0.139:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Readiness probe error: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Readiness probe failed: Get "http://10.133.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 3; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled down replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 5 to 3 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Deleted pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Deleted pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-d6kwd 45m Normal Killing pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-lzm9d 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-t9zcq 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Container created 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Add eth0 [10.132.0.108/23] from ovn-kubernetes 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Container started 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Add eth0 [10.133.0.101/23] from ovn-kubernetes 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-lnzgf Readiness probe failed: HTTP probe failed with statuscode: 503 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Container started 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.195s (4.195s including waiting). Image size: 185476477 bytes. 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Container created 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal AddedLabel gateway/e2e-shared-a-w5-d04acd Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-d04acd 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465 Created pod: e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-d04acd-openshift-default Scaled up replica set e2e-shared-a-w5-d04acd-openshift-default-6496b7b465 from 0 to 1 44m Normal AddedInterface pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Add eth0 [10.134.0.148/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-d04acd Scaled up replica set payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-d04acd Scaled up replica set payload-processing-e2e-shared-a-w5-d04acd-d55c97f55 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55 Created pod: payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-d04acd: serviceaccount "payload-processing-e2e-shared-a-w5-d04acd" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55 Created pod: payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55 Error creating: pods "payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-d04acd: serviceaccount "payload-processing-e2e-shared-a-w5-d04acd" not found 44m Normal ScalingReplicaSet deployment/e2e-mt-a7907d70-openshift-default Scaled up replica set e2e-mt-a7907d70-openshift-default-6d96ff57bf from 0 to 1 44m Normal AddedLabel gateway/e2e-mt-a7907d70 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a7907d70 44m Normal SuccessfulCreate replicaset/e2e-mt-a7907d70-openshift-default-6d96ff57bf Created pod: e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 Add eth0 [10.134.0.150/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 Container started 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 Add eth0 [10.134.0.149/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Add eth0 [10.134.0.151/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 Container created 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a7907d70 Scaled up replica set payload-pre-processing-e2e-mt-a7907d70-85d6dd5999 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-a7907d70-f46b98989 Error creating: pods "payload-processing-e2e-mt-a7907d70-f46b98989-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a7907d70: serviceaccount "payload-processing-e2e-mt-a7907d70" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a7907d70-f46b98989 Created pod: payload-processing-e2e-mt-a7907d70-f46b98989-sb697 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999 Error creating: pods "payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a7907d70: serviceaccount "payload-processing-e2e-mt-a7907d70" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a7907d70 Scaled up replica set payload-processing-e2e-mt-a7907d70-f46b98989 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999 Created pod: payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n 44m Normal Started pod/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n Add eth0 [10.134.0.152/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n Container started 44m Normal Pulled pod/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n Container created 44m Normal Created pod/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 Container created 44m Normal AddedInterface pod/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 Add eth0 [10.133.0.104/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-shared-a-w2-dc67dd Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-dc67dd 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c Created pod: e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz 44m Normal AddedInterface pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Add eth0 [10.134.0.154/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-dc67dd-openshift-default Scaled up replica set e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-dc67dd: serviceaccount "payload-processing-e2e-shared-a-w2-dc67dd" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-dc67dd Scaled up replica set payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-dc67dd Scaled up replica set payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f Error creating: pods "payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-dc67dd: serviceaccount "payload-processing-e2e-shared-a-w2-dc67dd" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479 Created pod: payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f Container created 44m Normal Created pod/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl Container created 44m Normal Pulled pod/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl Add eth0 [10.134.0.155/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f Add eth0 [10.132.0.109/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f Container started 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f Created pod: payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl 44m Normal Started pod/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl Container started 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a7907d70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a7907d70-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning ProbeError pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal Pulled pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Container created 43m Normal Started pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Container started 43m Normal Created pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Container created 43m Normal Pulled pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Container started 43m Warning ProbeError pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 43m Warning ProbeError pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c1bb9544-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c1bb9544-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Killing pod/payload-processing-e2e-ait-c1bb9544-7567c84675-xrsbj Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-ait-c1bb9544-55c58b8c4c-mtq48 Stopping container payload-pre-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ce4664d0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ce4664d0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j 43m Normal AddedLabel gateway/e2e-mt-a1b30a15 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a1b30a15 43m Normal ScalingReplicaSet deployment/e2e-mt-a1b30a15-openshift-default Scaled up replica set e2e-mt-a1b30a15-openshift-default-55cc89b84f from 0 to 1 43m Normal Killing pod/payload-processing-e2e-ait-ce4664d0-56c8dc8f55-k7bzh Stopping container payload-processing 43m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Add eth0 [10.133.0.105/23] from ovn-kubernetes 43m Normal Killing pod/payload-pre-processing-e2e-ait-ce4664d0-76b44c4f9f-8xs6q Stopping container payload-pre-processing 43m Normal Started pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Container started 43m Normal Pulled pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Container created 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c1bb9544-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c1bb9544-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a1b30a15 Scaled up replica set payload-processing-e2e-mt-a1b30a15-77d5dd87f9 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779 Error creating: pods "payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a1b30a15: serviceaccount "payload-processing-e2e-mt-a1b30a15" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-a1b30a15-77d5dd87f9 Error creating: pods "payload-processing-e2e-mt-a1b30a15-77d5dd87f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a1b30a15: serviceaccount "payload-processing-e2e-mt-a1b30a15" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a1b30a15 Scaled up replica set payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779 Created pod: payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a1b30a15-77d5dd87f9 Created pod: payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm 43m Normal Pulled pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Container started 43m Normal Created pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Container created 43m Normal Started pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Container started 43m Normal AddedInterface pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Add eth0 [10.133.0.107/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Add eth0 [10.133.0.106/23] from ovn-kubernetes 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-d04acd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-d04acd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Created pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Container created 42m Normal Pulled pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Container started 42m Normal Killing pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-shared-a-w0-6b1cca Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-6b1cca 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-d04acd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-d04acd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning ProbeError pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-ait-c1bb9544-openshift-default-6467754d77-9qhzc Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc Created pod: e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-6b1cca-openshift-default Scaled up replica set e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc from 0 to 1 42m Normal AddedInterface pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Add eth0 [10.133.0.108/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 42m Normal Killing pod/e2e-ait-ce4664d0-openshift-default-5c55fff65c-sn7tw Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-shared-a-w4-822dd4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-822dd4 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-822dd4-openshift-default Scaled up replica set e2e-shared-a-w4-822dd4-openshift-default-854cd7994c from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c Created pod: e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq 42m Normal AddedLabel gateway/e2e-api-a-dac4fa Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-dac4fa 42m Normal SuccessfulCreate replicaset/e2e-api-a-dac4fa-openshift-default-7c99579dbc Created pod: e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs 42m Normal AddedLabel gateway/e2e-shared-b-w5-f43174 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-f43174 42m Normal ScalingReplicaSet deployment/e2e-api-a-dac4fa-openshift-default Scaled up replica set e2e-api-a-dac4fa-openshift-default-7c99579dbc from 0 to 1 42m Normal AddedInterface pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Add eth0 [10.134.0.158/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-f43174-openshift-default Scaled up replica set e2e-shared-b-w5-f43174-openshift-default-5f7c77757 from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff 42m Normal AddedInterface pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Add eth0 [10.133.0.109/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Add eth0 [10.134.0.159/23] from ovn-kubernetes 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc Error creating: pods "payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-6b1cca: serviceaccount "payload-processing-e2e-shared-a-w0-6b1cca" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc Created pod: payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-6b1cca Scaled up replica set payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-6b1cca: serviceaccount "payload-processing-e2e-shared-a-w0-6b1cca" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9 Created pod: payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-6b1cca Scaled up replica set payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9 from 0 to 1 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 Add eth0 [10.133.0.110/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 Container created 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl 42m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 5; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled down replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 10 to 5 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz 42m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Stopping container istio-proxy 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f Add eth0 [10.133.0.111/23] from ovn-kubernetes 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq 42m Normal Pulled pod/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf 42m Normal Created pod/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f Container created 42m Normal Started pod/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f Container started 42m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled down replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 10 to 5 42m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-v2nnf Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r 42m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-22w7r Stopping container istio-proxy 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 5; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-89bcx 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 Container started 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-2mtxp 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 5; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-68tlq Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv 42m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-95wnl Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fqnvv 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb 42m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-wxsbb Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-8b7jd Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled down replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 10 to 5 42m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc 42m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-5b2hv Stopping container istio-proxy 42m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-hc5lj Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-822dd4 Scaled up replica set payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c Created pod: payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc 42m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-ldg9x Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69 Created pod: payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c Error creating: pods "payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-822dd4: serviceaccount "payload-processing-e2e-shared-a-w4-822dd4" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69 Error creating: pods "payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-822dd4: serviceaccount "payload-processing-e2e-shared-a-w4-822dd4" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-822dd4 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c from 0 to 1 42m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-b9ls9 Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-dac4fa Scaled up replica set payload-processing-e2e-api-a-dac4fa-6dc74cb485 from 0 to 1 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc Add eth0 [10.133.0.112/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc Container started 42m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk Add eth0 [10.133.0.113/23] from ovn-kubernetes 42m Normal Pulled pod/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk Container created 42m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc Container created 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-dac4fa Scaled up replica set payload-pre-processing-e2e-api-a-dac4fa-576669c89f from 0 to 1 42m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-dac4fa-6dc74cb485 Error creating: pods "payload-processing-e2e-api-a-dac4fa-6dc74cb485-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-dac4fa: serviceaccount "payload-processing-e2e-api-a-dac4fa" not found 42m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Readiness probe failed: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-dac4fa-6dc74cb485 Created pod: payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-dac4fa-576669c89f Error creating: pods "payload-pre-processing-e2e-api-a-dac4fa-576669c89f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-dac4fa: serviceaccount "payload-processing-e2e-api-a-dac4fa" not found 42m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-l9fmz Readiness probe error: Get "http://10.133.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Started pod/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk Container started 42m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-hgnhc Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-dac4fa-576669c89f Created pod: payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 42m Normal Started pod/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 Container started 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 Add eth0 [10.133.0.115/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 Container created 42m Normal Created pod/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 Container created 42m Normal Pulled pod/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 Add eth0 [10.133.0.114/23] from ovn-kubernetes 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-f43174: serviceaccount "payload-processing-e2e-shared-b-w5-f43174" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47 Created pod: payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf Created pod: payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf Error creating: pods "payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-f43174: serviceaccount "payload-processing-e2e-shared-b-w5-f43174" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-f43174 Scaled up replica set payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-f43174 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-f43174-56964f47 from 0 to 1 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 Container started 42m Normal Created pod/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx Container created 42m Normal Pulled pod/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx Add eth0 [10.133.0.117/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 Add eth0 [10.133.0.116/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 Container created 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a7907d70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a7907d70-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal Killing pod/payload-processing-e2e-mt-a7907d70-f46b98989-sb697 Stopping container payload-processing 42m Normal Started pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Container started 42m Normal Pulled pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Container created 42m Normal Killing pod/payload-pre-processing-e2e-mt-a7907d70-85d6dd5999-2c79n Stopping container payload-pre-processing 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-dc67dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-dc67dd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning ProbeError pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Startup probe error: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused... 42m Normal AddedLabel gateway/e2e-shared-b-w2-705a67 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-705a67 42m Warning Unhealthy pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Startup probe failed: Get "http://10.134.0.154:15021/healthz/ready": dial tcp 10.134.0.154:15021: connect: connection refused 42m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-705a67-openshift-default-85c774bd65 Created pod: e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-705a67-openshift-default Scaled up replica set e2e-shared-b-w2-705a67-openshift-default-85c774bd65 from 0 to 1 42m Normal AddedInterface pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Add eth0 [10.133.0.119/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-705a67 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-705a67 Scaled up replica set payload-processing-e2e-shared-b-w2-705a67-868484c from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-705a67-868484c Created pod: payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c Created pod: payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-705a67-868484c Error creating: pods "payload-processing-e2e-shared-b-w2-705a67-868484c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-705a67: serviceaccount "payload-processing-e2e-shared-b-w2-705a67" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c Error creating: pods "payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-705a67: serviceaccount "payload-processing-e2e-shared-b-w2-705a67" not found 41m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth Add eth0 [10.133.0.125/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh Container started 41m Normal Created pod/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh Container created 41m Normal Pulled pod/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh Add eth0 [10.133.0.126/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth Container started 41m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth Container created 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 7; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled down replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 10 to 7 41m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-xxlfm Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-zx64r Stopping container istio-proxy 41m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Readiness probe error: HTTP probe failed with statuscode: 503... 41m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled down replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 10 to 7 41m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 7; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-cln4k 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-99qrc 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled down replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 10 to 7 41m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-h7pbt Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-lj74w Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-r9rlj Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-dmssv 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 7; reason: All metrics below target 41m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-v5q8h Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-dc67dd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-dc67dd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Readiness probe error: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Readiness probe failed: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Container started 41m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Container created 41m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 41m Normal Killing pod/e2e-mt-a7907d70-openshift-default-6d96ff57bf-n52c7 Stopping container istio-proxy 41m Normal AddedLabel gateway/e2e-api-a-e977aa Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-e977aa 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal ScalingReplicaSet deployment/e2e-api-a-e977aa-openshift-default Scaled up replica set e2e-api-a-e977aa-openshift-default-67bb57797 from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-a-e977aa-openshift-default-67bb57797 Created pod: e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 41m Normal AddedInterface pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Add eth0 [10.133.0.130/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-e977aa Scaled up replica set payload-processing-e2e-api-a-e977aa-bbc97bd5f from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-e977aa-bbc97bd5f Created pod: payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-e977aa Scaled up replica set payload-pre-processing-e2e-api-a-e977aa-769db8bb9f from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f Created pod: payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f Error creating: pods "payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e977aa: serviceaccount "payload-processing-e2e-api-a-e977aa" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-e977aa-bbc97bd5f Error creating: pods "payload-processing-e2e-api-a-e977aa-bbc97bd5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-e977aa: serviceaccount "payload-processing-e2e-api-a-e977aa" not found 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 Add eth0 [10.133.0.132/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 Add eth0 [10.133.0.133/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 Container started 41m Normal Created pod/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 Container created 41m Normal Started pod/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 Container started 41m Normal Pulled pod/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 Container created 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Created pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Container created 41m Normal Started pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Container started 41m Normal Pulled pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 41m Warning ProbeError pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-6b1cca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-6b1cca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-6b1cca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-6b1cca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn 40m Normal Pulled pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv 40m Normal Started pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Container started 40m Normal Created pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Container created 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-f43174-openshift-default Scaled up replica set e2e-shared-b-w5-f43174-openshift-default-5f7c77757 from 1 to 4 40m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Container started 40m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Container started 40m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Container created 40m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Container started 40m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Readiness probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 40m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Add eth0 [10.133.0.136/23] from ovn-kubernetes 40m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Container created 40m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Add eth0 [10.132.0.113/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Readiness probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 40m Normal Started pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Container started 40m Normal Created pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Container created 40m Normal Pulled pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Container created 40m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Container started 40m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Container created 40m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Add eth0 [10.133.0.137/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Startup probe error: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Startup probe failed: Get "http://10.134.0.158:15021/healthz/ready": dial tcp 10.134.0.158:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Startup probe error: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Startup probe failed: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-822dd4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-822dd4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-dac4fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-dac4fa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal AddedLabel gateway/e2e-shared-b-w0-487213 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-487213 40m Normal AddedInterface pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Add eth0 [10.134.0.168/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7 Created pod: e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-487213-openshift-default Scaled up replica set e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7 from 0 to 1 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled down replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 5 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 4; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled down replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 5 to 4 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 4; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 4; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-8569j 40m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-d4cps Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled down replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 5 to 4 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb 40m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-d0755f-openshift-default Scaled up replica set e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c from 0 to 1 40m Normal AddedLabel gateway/e2e-shared-b-w4-d0755f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-d0755f 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c Created pod: e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb 40m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-k5grb Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal AddedInterface pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Add eth0 [10.134.0.169/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-8569j Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal AddedLabel gateway/e2e-api-b-3dc1da Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-3dc1da 40m Normal ScalingReplicaSet deployment/e2e-api-b-3dc1da-openshift-default Scaled up replica set e2e-api-b-3dc1da-openshift-default-64679f6975 from 0 to 1 40m Normal SuccessfulCreate replicaset/e2e-api-b-3dc1da-openshift-default-64679f6975 Created pod: e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-822dd4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-487213 Scaled up replica set payload-processing-e2e-shared-b-w0-487213-7795765697 from 0 to 1 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-822dd4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedInterface pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Add eth0 [10.134.0.170/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-487213 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9 from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-487213: serviceaccount "payload-processing-e2e-shared-b-w0-487213" not found 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-dac4fa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-487213-7795765697 Created pod: payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-487213-7795765697 Error creating: pods "payload-processing-e2e-shared-b-w0-487213-7795765697-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-487213: serviceaccount "payload-processing-e2e-shared-b-w0-487213" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9 Created pod: payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-dac4fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Created pod/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 Container created 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj Container created 39m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj Add eth0 [10.133.0.138/23] from ovn-kubernetes 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj Container started 39m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 Add eth0 [10.133.0.139/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 Container started 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-d0755f Scaled up replica set payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-d0755f Scaled up replica set payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc Error creating: pods "payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-d0755f: serviceaccount "payload-processing-e2e-shared-b-w4-d0755f" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc Created pod: payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk 39m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc Error creating: pods "payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-d0755f: serviceaccount "payload-processing-e2e-shared-b-w4-d0755f" not found 39m Normal AddedLabel gateway/e2e-api-a-bdc828 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-bdc828 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc Created pod: payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm 39m Normal Started pod/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm Add eth0 [10.133.0.141/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm Container started 39m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm Container created 39m Normal Created pod/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk Container created 39m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk Add eth0 [10.133.0.142/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Warning FailedCreate replicaset/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7 Error creating: pods "payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3dc1da: serviceaccount "payload-processing-e2e-api-b-3dc1da" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7 Created pod: payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f Error creating: pods "payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-3dc1da: serviceaccount "payload-processing-e2e-api-b-3dc1da" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f Created pod: payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb 39m Normal AddedInterface pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Add eth0 [10.134.0.171/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-api-a-bdc828-openshift-default-569d5846c Created pod: e2e-api-a-bdc828-openshift-default-569d5846c-grp6f 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-3dc1da Scaled up replica set payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f from 0 to 1 39m Normal ScalingReplicaSet deployment/e2e-api-a-bdc828-openshift-default Scaled up replica set e2e-api-a-bdc828-openshift-default-569d5846c from 0 to 1 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-3dc1da Scaled up replica set payload-processing-e2e-api-b-3dc1da-6b9b59d4c7 from 0 to 1 39m Normal Pulled pod/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb Add eth0 [10.132.0.114/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 Add eth0 [10.133.0.143/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb Container started 39m Normal Started pod/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 Container started 39m Normal Created pod/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 Container created 39m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Add eth0 [10.132.0.115/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default New size: 8; reason: All metrics below target 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln 39m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-f43174-openshift-default Scaled up replica set e2e-shared-b-w5-f43174-openshift-default-5f7c77757 from 4 to 8 39m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w 39m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Add eth0 [10.134.0.173/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Created pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 39m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Add eth0 [10.133.0.144/23] from ovn-kubernetes 39m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Container created 39m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Container started 39m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Container created 39m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Container started 39m Normal Created pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Container created 39m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Container created 39m Normal Pulled pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Add eth0 [10.134.0.172/23] from ovn-kubernetes 39m Normal Created pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Container created 39m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Container started 39m Normal Pulled pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Container started 39m Normal Started pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Container started 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-bdc828 Scaled up replica set payload-processing-e2e-api-a-bdc828-647975f48f from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-bdc828-546768c7f5 Created pod: payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-bdc828-647975f48f Error creating: pods "payload-processing-e2e-api-a-bdc828-647975f48f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-bdc828: serviceaccount "payload-processing-e2e-api-a-bdc828" not found 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-bdc828-546768c7f5 Error creating: pods "payload-pre-processing-e2e-api-a-bdc828-546768c7f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-bdc828: serviceaccount "payload-processing-e2e-api-a-bdc828" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-bdc828 Scaled up replica set payload-pre-processing-e2e-api-a-bdc828-546768c7f5 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-bdc828-647975f48f Created pod: payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv 39m Normal AddedInterface pod/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv Add eth0 [10.132.0.116/23] from ovn-kubernetes 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Pulled pod/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 Add eth0 [10.133.0.145/23] from ovn-kubernetes 39m Normal Created pod/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv Container created 39m Normal Started pod/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv Container started 39m Normal Created pod/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 Container created 39m Normal Started pod/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 Container started 39m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Startup probe failed: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused 39m Warning Unhealthy pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 39m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Startup probe error: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3dc1da-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3dc1da-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-705a67-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-705a67-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-bdc828-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-bdc828-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal ScalingReplicaSet deployment/e2e-inf-a-a2ae37-openshift-default Scaled up replica set e2e-inf-a-a2ae37-openshift-default-67d57d5d77 from 0 to 1 39m Normal AddedLabel gateway/e2e-inf-a-a2ae37 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a2ae37 39m Normal SuccessfulCreate replicaset/e2e-inf-a-a2ae37-openshift-default-67d57d5d77 Created pod: e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 39m Normal AddedInterface pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Add eth0 [10.134.0.174/23] from ovn-kubernetes 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-705a67-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-705a67-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a2ae37 Scaled up replica set payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c from 0 to 1 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f Error creating: pods "payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a2ae37: serviceaccount "payload-processing-e2e-inf-a-a2ae37" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a2ae37 Scaled up replica set payload-pre-processing-e2e-inf-a-a2ae37-744c88897f from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c Error creating: pods "payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a2ae37: serviceaccount "payload-processing-e2e-inf-a-a2ae37" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c Created pod: payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr Add eth0 [10.132.0.117/23] from ovn-kubernetes 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f Created pod: payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr 39m Normal AddedInterface pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Add eth0 [10.132.0.118/23] from ovn-kubernetes 39m Normal Pulled pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Created pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Container created 39m Normal Started pod/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr Container started 39m Normal Created pod/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr Container created 39m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Container started 38m Normal Pulled pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Container created 38m Normal Started pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Container started 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e977aa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e977aa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning Unhealthy pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Startup probe failed: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Startup probe error: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused... 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-e977aa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-e977aa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Killing pod/payload-processing-e2e-api-a-e977aa-bbc97bd5f-swn99 Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-api-a-e977aa-769db8bb9f-9bnh8 Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Stopping container payload-pre-processing 38m Warning Unhealthy pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Readiness probe failed: Get "http://10.134.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Readiness probe error: Get "http://10.134.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Created pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Container created 38m Normal Started pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Container started 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 38m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-zgr9j Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-zgr9j 38m Normal Pulled pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-487213-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-487213-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning Unhealthy pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-4cxlm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-6zhpm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-487213-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-487213-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default Scaled down replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 4 to 3 37m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-stllq 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 3; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 3; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 3; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 6; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled down replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 7 to 6 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz 37m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled down replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 4 to 3 37m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x 37m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default Scaled down replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 4 to 3 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd 37m Normal Killing pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Deleted pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled down replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 7 to 6 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 6; reason: All metrics below target 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 6; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc 37m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled down replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 7 to 6 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-nbw47 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 37m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-nbw47 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled down replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 3 to 2 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 2; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-lzm9d 37m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-lzm9d Stopping container discovery 37m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 37m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-884jd Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 37m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-pqrsz Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w0-ac1273-openshift-default-79699df87-stllq Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Readiness probe error: Get "http://10.134.0.169:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Readiness probe failed: Get "http://10.134.0.169:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-bqrp8 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-z7gfh Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-65jtc Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-cj29x Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal Pulled pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Container created 37m Normal Created pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Container created 37m Normal Started pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Container started 37m Normal SuccessfulCreate replicaset/e2e-api-a-bdc828-openshift-default-569d5846c Created pod: e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-a-bdc828-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-api-a-bdc828-openshift-default-569d5846c Created pod: e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 37m Normal Started pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Container started 37m Normal Created pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Container created 37m Normal AddedInterface pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Add eth0 [10.133.0.148/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/e2e-api-a-bdc828-openshift-default Scaled up replica set e2e-api-a-bdc828-openshift-default-569d5846c from 1 to 3 37m Normal Pulled pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Container started 37m Normal Pulled pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Pulled pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Container started 37m Normal Created pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Container created 37m Normal Started pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Container started 37m Normal AddedInterface pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Add eth0 [10.132.0.123/23] from ovn-kubernetes 37m Normal Created pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Container created 37m Normal Pulled pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Readiness probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 37m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-mb94r Readiness probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Startup probe failed: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused 37m Warning Unhealthy pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Startup probe failed: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Startup probe error: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused 37m Warning ProbeError pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Startup probe error: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused... 37m Warning ProbeError pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Startup probe error: Get "http://10.134.0.169:15021/healthz/ready": dial tcp 10.134.0.169:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Startup probe failed: Get "http://10.134.0.170:15021/healthz/ready": dial tcp 10.134.0.170:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled down replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 6 to 4 37m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-jqtwq 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 5; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled down replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 6 to 5 37m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-hqc84 Stopping container istio-proxy 37m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Stopping container istio-proxy 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-d0755f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-c6nct Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 4; reason: All metrics below target 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-d0755f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk 37m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled down replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 6 to 4 37m Warning Unhealthy pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p58mg Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-rw2sk Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-ggxdf Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-3dc1da-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-3dc1da-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal Killing pod/payload-processing-e2e-shared-a-w0-6b1cca-5c4798f5cc-7mt2f Stopping container payload-processing 36m Warning ProbeError pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Readiness probe error: Get "http://10.134.0.174:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Readiness probe failed: Get "http://10.134.0.174:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-6b1cca-5f6fcfc8c9-g5hl9 Stopping container payload-pre-processing 36m Normal Pulled pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Container created 36m Normal Started pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Container started 36m Normal Killing pod/payload-processing-e2e-shared-a-w4-822dd4-666b4c4c69-87ntk Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-822dd4-74865cd46c-7kbdc Stopping container payload-pre-processing 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-bdc828-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-bdc828-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-pre-processing-e2e-api-a-dac4fa-576669c89f-wqcw8 Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-a-dac4fa-6dc74cb485-skh76 Stopping container payload-processing 36m Warning ProbeError pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Startup probe error: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Startup probe failed: Get "http://10.134.0.174:15021/healthz/ready": dial tcp 10.134.0.174:15021: connect: connection refused 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a2ae37-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a2ae37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-d0755f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-d0755f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/payload-pre-processing-e2e-api-a-bdc828-546768c7f5-vjkp7 Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-a-bdc828-647975f48f-wvqlv Stopping container payload-processing 36m Normal Killing pod/e2e-api-a-e977aa-openshift-default-67bb57797-kfhv9 Stopping container istio-proxy 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a2ae37-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a2ae37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 36m Normal AddedLabel gateway/e2e-inf-a-05db63 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-05db63 36m Normal SuccessfulCreate replicaset/e2e-inf-a-05db63-openshift-default-8cdbcbbdc Created pod: e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 36m Normal ScalingReplicaSet deployment/e2e-inf-a-05db63-openshift-default Scaled up replica set e2e-inf-a-05db63-openshift-default-8cdbcbbdc from 0 to 1 36m Normal AddedInterface pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Add eth0 [10.133.0.149/23] from ovn-kubernetes 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-05db63 Scaled up replica set payload-processing-e2e-inf-a-05db63-6ddf9bc6f7 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7 Created pod: payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7 Error creating: pods "payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-05db63: serviceaccount "payload-processing-e2e-inf-a-05db63" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f Error creating: pods "payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-05db63: serviceaccount "payload-processing-e2e-inf-a-05db63" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f Created pod: payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-05db63 Scaled up replica set payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f from 0 to 1 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Add eth0 [10.133.0.151/23] from ovn-kubernetes 36m Normal Started pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Add eth0 [10.133.0.150/23] from ovn-kubernetes 36m Normal Created pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Container created 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Container started 36m Normal Created pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Container created 36m Normal Pulled pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Stopping container istio-proxy 36m Normal Killing pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Stopping container istio-proxy 36m Normal Killing pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Stopping container istio-proxy 36m Warning ProbeError pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-api-a-bdc828-openshift-default-569d5846c-jhr9l Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-api-a-bdc828-openshift-default-569d5846c-grp6f Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-api-a-bdc828-openshift-default-569d5846c-gfsl2 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal AddedLabel gateway/e2e-inf-a-636a7f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-636a7f 36m Normal AddedInterface pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Add eth0 [10.133.0.152/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-inf-a-636a7f-openshift-default-cc597bf49 Created pod: e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 36m Normal ScalingReplicaSet deployment/e2e-inf-a-636a7f-openshift-default Scaled up replica set e2e-inf-a-636a7f-openshift-default-cc597bf49 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-636a7f Scaled up replica set payload-processing-e2e-inf-a-636a7f-9859998f9 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447 Created pod: payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-636a7f-9859998f9 Created pod: payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447 Error creating: pods "payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-636a7f: serviceaccount "payload-processing-e2e-inf-a-636a7f" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-636a7f Scaled up replica set payload-pre-processing-e2e-inf-a-636a7f-c7dd99447 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-636a7f-9859998f9 Error creating: pods "payload-processing-e2e-inf-a-636a7f-9859998f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-636a7f: serviceaccount "payload-processing-e2e-inf-a-636a7f" not found 36m Normal Pulled pod/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz Container created 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz Add eth0 [10.133.0.153/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz Container started 36m Normal Started pod/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb Container started 36m Normal Created pod/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb Container created 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb Add eth0 [10.133.0.154/23] from ovn-kubernetes 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-a2ae37-744c88897f-hj5sr Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Stopping container payload-processing 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-05db63-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-05db63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedCreate replicaset/payload-processing-e2e-mt-a1b30a15-77d5dd87f9 Error creating: pods "payload-processing-e2e-mt-a1b30a15-77d5dd87f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a1b30a15: serviceaccount "payload-processing-e2e-mt-a1b30a15" not found 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a1b30a15-77d5dd87f9 Created pod: payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a1b30a15 Scaled up replica set payload-processing-e2e-mt-a1b30a15-77d5dd87f9 from 0 to 1 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779 Error creating: pods "payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a1b30a15: serviceaccount "payload-processing-e2e-mt-a1b30a15" not found 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a1b30a15 Scaled up replica set payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779 Created pod: payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 35m Normal Started pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx Container started 35m Normal Pulled pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx Add eth0 [10.133.0.156/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 Container created 35m Normal Started pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 Container started 35m Normal Created pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 Add eth0 [10.133.0.155/23] from ovn-kubernetes 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-a2ae37-7bc5cc6b4c-87d5l Readiness probe failed: timeout: failed to connect service "10.132.0.118:50052" within 1s: context deadline exceeded 35m Normal Killing pod/e2e-inf-a-a2ae37-openshift-default-67d57d5d77-qjgr4 Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Deleted pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default New size: 7; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-f43174-openshift-default Scaled down replica set e2e-shared-b-w5-f43174-openshift-default-5f7c77757 from 8 to 7 35m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-lwfw2 Stopping container istio-proxy 35m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-dc67dd-d48ffc479-shb7f Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-a-w2-dc67dd-5859967d4f-p2prl Stopping container payload-processing 35m Normal Killing pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Stopping container istio-proxy 35m Normal Killing pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Stopping container istio-proxy 35m Warning ProbeError pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-shared-a-w4-822dd4-openshift-default-854cd7994c-hj9qq Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-shared-a-w0-6b1cca-openshift-default-644dc8c7bc-jsjzh Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/payload-processing-e2e-shared-b-w0-487213-7795765697-cdgg8 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-487213-869b67d7f9-ddqrj Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-shared-b-w4-d0755f-d59ff87cc-phddk Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-d0755f-dc9b99cbc-rt7tm Stopping container payload-pre-processing 34m Normal Killing pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-shared-a-w2-dc67dd-openshift-default-587b579d8c-d98fz Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 3; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 2; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 2; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-4fe3a4-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 from 3 to 2 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77 Deleted pod: e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 34m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-rg6fp 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-4fe3a4-openshift-default New size: 2; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default Scaled down replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 3 to 2 34m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-p4z24 34m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-vggc8 Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default Scaled down replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 4 to 3 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default Scaled down replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 5 to 4 34m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-t95p5 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-9v9ps Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/payload-processing-e2e-shared-b-w2-705a67-868484c-7r6wh Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-705a67-689d9d764c-jdfth Stopping container payload-pre-processing 34m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Deleted pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn 34m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-gp2jn Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-f43174-openshift-default Scaled down replica set e2e-shared-b-w5-f43174-openshift-default-5f7c77757 from 7 to 4 34m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Deleted pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-t8f9c 34m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Deleted pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-l7bmv 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-tc5qb 34m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 34m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-t9zcq Stopping container discovery 34m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-t9zcq 34m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 34m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-tc5qb Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal Started pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Container started 34m Normal Created pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Container created 34m Normal Pulled pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning ProbeError pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 33m Normal ScalingReplicaSet deployment/e2e-mt-a1b30a15-openshift-default Scaled up replica set e2e-mt-a1b30a15-openshift-default-55cc89b84f from 1 to 4 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Add eth0 [10.132.0.124/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Add eth0 [10.133.0.160/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Add eth0 [10.134.0.187/23] from ovn-kubernetes 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Container created 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Container started 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Container created 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Container started 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Container created 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Container started 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Killing pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Stopping container istio-proxy 33m Normal Killing pod/e2e-shared-b-w0-487213-openshift-default-6bbb9d74d7-ffhdp Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-api-a-dac4fa-openshift-default-7c99579dbc-mzdgs Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Startup probe error: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Startup probe failed: Get "http://10.133.0.160:15021/healthz/ready": dial tcp 10.133.0.160:15021: connect: connection refused 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Startup probe error: Get "http://10.134.0.187:15021/healthz/ready": dial tcp 10.134.0.187:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Startup probe failed: Get "http://10.134.0.187:15021/healthz/ready": dial tcp 10.134.0.187:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 33m Normal Killing pod/e2e-shared-b-w4-d0755f-openshift-default-69b4777c4c-qrzxb Stopping container istio-proxy 33m Normal Killing pod/payload-processing-e2e-api-b-3dc1da-6b9b59d4c7-wfxb9 Stopping container payload-processing 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default New size: 8; reason: 33m Normal ScalingReplicaSet deployment/e2e-mt-a1b30a15-openshift-default Scaled up replica set e2e-mt-a1b30a15-openshift-default-55cc89b84f from 4 to 8 33m Normal Killing pod/payload-pre-processing-e2e-api-b-3dc1da-5bdd65f95f-hlgpb Stopping container payload-pre-processing 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Container started 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Container created 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-05db63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-05db63-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Container started 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Container created 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Container started 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Add eth0 [10.132.0.125/23] from ovn-kubernetes 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Container created 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Add eth0 [10.134.0.191/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Add eth0 [10.134.0.190/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Add eth0 [10.133.0.161/23] from ovn-kubernetes 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Container started 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Container created 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Startup probe failed: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Startup probe error: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Startup probe failed: Get "http://10.133.0.161:15021/healthz/ready": dial tcp 10.133.0.161:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Startup probe error: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused... 33m Normal AddedLabel gateway/e2e-inf-b-66db1c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-66db1c 33m Normal SuccessfulCreate replicaset/e2e-inf-b-66db1c-openshift-default-798cbbdd45 Created pod: e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w 33m Normal ScalingReplicaSet deployment/e2e-inf-b-66db1c-openshift-default Scaled up replica set e2e-inf-b-66db1c-openshift-default-798cbbdd45 from 0 to 1 33m Normal AddedInterface pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Add eth0 [10.133.0.162/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f (combined from similar events): Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz 33m Normal ScalingReplicaSet deployment/e2e-mt-a1b30a15-openshift-default Scaled up replica set e2e-mt-a1b30a15-openshift-default-55cc89b84f from 8 to 10 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-a1b30a15-openshift-default New size: 10; reason: 33m Normal SuccessfulCreate replicaset/e2e-mt-a1b30a15-openshift-default-55cc89b84f Created pod: e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Container created 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Add eth0 [10.133.0.163/23] from ovn-kubernetes 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Container started 33m Normal AddedInterface pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Add eth0 [10.132.0.126/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Container started 33m Normal Created pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Container created 33m Warning Unhealthy pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Readiness probe failed: Get "http://10.133.0.152:15021/healthz/ready": read tcp 10.133.0.2:43546->10.133.0.152:15021: read: connection reset by peer 33m Normal Started pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Container started 33m Warning ProbeError pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Readiness probe error: Get "http://10.133.0.152:15021/healthz/ready": read tcp 10.133.0.2:43546->10.133.0.152:15021: read: connection reset by peer... 33m Normal Pulled pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Container created 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 33m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-66db1c Scaled up replica set payload-processing-e2e-inf-b-66db1c-77f86d9bbc from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-66db1c Scaled up replica set payload-pre-processing-e2e-inf-b-66db1c-d95868c6c from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-66db1c-77f86d9bbc Created pod: payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-66db1c-77f86d9bbc Error creating: pods "payload-processing-e2e-inf-b-66db1c-77f86d9bbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-66db1c: serviceaccount "payload-processing-e2e-inf-b-66db1c" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c Created pod: payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c Error creating: pods "payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-66db1c: serviceaccount "payload-processing-e2e-inf-b-66db1c" not found 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Add eth0 [10.133.0.164/23] from ovn-kubernetes 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Add eth0 [10.133.0.165/23] from ovn-kubernetes 33m Normal Created pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Container created 33m Normal Started pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Container started 33m Normal Pulled pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Container created 33m Normal Started pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Container started 33m Warning Unhealthy pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 33m Normal Killing pod/e2e-shared-b-w2-705a67-openshift-default-85c774bd65-pnlgr Stopping container istio-proxy 32m Normal Killing pod/payload-pre-processing-e2e-mt-a1b30a15-65bcd6b779-pjxz2 Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-mt-a1b30a15-77d5dd87f9-vzvhx Stopping container payload-processing 32m Normal Killing pod/payload-processing-e2e-inf-a-636a7f-9859998f9-6fstb Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-636a7f-c7dd99447-zf9bz Stopping container payload-pre-processing 32m Normal Killing pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Stopping container payload-processing 32m Normal ScalingReplicaSet deployment/e2e-worker-w2-e7f13a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-e7f13a-openshift-default-d9477795f from 4 to 3 32m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s Stopping container istio-proxy 32m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n 32m Normal SuccessfulDelete replicaset/e2e-worker-w5-ef6d93-openshift-default-58df979549 Deleted pod: e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 32m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w2-e7f13a-openshift-default-d9477795f Deleted pod: e2e-worker-w2-e7f13a-openshift-default-d9477795f-k2f8s 32m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-fcb75c45f from 5 to 3 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-ef6d93-openshift-default New size: 2; reason: All metrics below target 32m Normal ScalingReplicaSet deployment/e2e-worker-w5-ef6d93-openshift-default Scaled down replica set e2e-worker-w5-ef6d93-openshift-default-58df979549 from 4 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-e7f13a-openshift-default New size: 3; reason: All metrics below target 32m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-crcp6 Stopping container istio-proxy 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-sjzsr Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-t7qph Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-jkb4n Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-636a7f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-636a7f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Deleted pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-wzcpk 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default New size: 2; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Stopping container istio-proxy 32m Warning ProbeError pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal ScalingReplicaSet deployment/e2e-worker-w4-59357a-openshift-default Scaled down replica set e2e-worker-w4-59357a-openshift-default-5b84865d9c from 2 to 1 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-59357a-openshift-default New size: 1; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w4-59357a-openshift-default-5b84865d9c Deleted pod: e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 32m Warning Unhealthy pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-k2mk5 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Killing pod/e2e-api-b-3dc1da-openshift-default-64679f6975-6lz7f Stopping container istio-proxy 32m Warning Unhealthy pod/payload-processing-e2e-worker-w2-e7f13a-687d7c9f45-wt5d4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-e7f13a-fff8d7787-f82q9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Stopping container payload-pre-processing 32m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 32m Normal Killing pod/payload-processing-e2e-worker-w1-4fe3a4-6474577947-xj4nh Stopping container payload-processing 32m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-fcb75c45f from 3 to 6 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Add eth0 [10.132.0.128/23] from ovn-kubernetes 32m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Container created 32m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Container created 32m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Add eth0 [10.134.0.198/23] from ovn-kubernetes 32m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Container started 32m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Container created 32m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Container started 32m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Container started 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Add eth0 [10.133.0.167/23] from ovn-kubernetes 32m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Startup probe error: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused... 32m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 32m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Startup probe error: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused... 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Startup probe failed: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Startup probe failed: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused 32m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 32m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Container started 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Add eth0 [10.134.0.199/23] from ovn-kubernetes 32m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Container created 32m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Container started 32m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Add eth0 [10.133.0.168/23] from ovn-kubernetes 32m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Container created 32m Normal Killing pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-inf-a-636a7f-openshift-default-cc597bf49-fgmh8 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Startup probe failed: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused 31m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Startup probe error: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused... 31m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-4fe3a4-54d4d5b77f-vshw5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Startup probe failed: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused 31m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Startup probe error: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Readiness probe failed: Get "http://10.133.0.162:15021/healthz/ready": read tcp 10.133.0.2:45490->10.133.0.162:15021: read: connection reset by peer 31m Warning ProbeError pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Readiness probe error: Get "http://10.133.0.162:15021/healthz/ready": read tcp 10.133.0.2:45490->10.133.0.162:15021: read: connection reset by peer... 31m Normal Created pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Container created 31m Normal Pulled pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Container started 31m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-d04acd-8c7fdcf55-6gff2 Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-shared-a-w5-d04acd-d55c97f55-pp6z2 Stopping container payload-processing 31m Warning Unhealthy pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Startup probe failed: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Startup probe error: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused... 31m Normal Killing pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Stopping container payload-processing 31m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vnmp2 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-vvx8g Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5bq65 Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-gvzj8 Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-5hk24 Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-xfb7q Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-smckv Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-9ms2h Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-qxxrw Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Stopping container istio-proxy 31m Normal Killing pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-khjnh Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-ht7mz Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-mt-a1b30a15-openshift-default-55cc89b84f-dxd7j Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w2-e7f13a-openshift-default-d9477795f-xsxhv Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-66db1c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-66db1c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning Unhealthy pod/payload-processing-e2e-worker-w4-59357a-7df684b4b7-wbbls Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-59357a-88f6c5975-dm79j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Add eth0 [10.133.0.169/23] from ovn-kubernetes 31m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Container created 31m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Container started 31m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Container created 31m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Container started 31m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Add eth0 [10.132.0.129/23] from ovn-kubernetes 30m Normal Killing pod/e2e-shared-a-w5-d04acd-openshift-default-6496b7b465-c28vp Stopping container istio-proxy 30m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Startup probe error: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused... 30m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Startup probe error: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused... 30m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Startup probe failed: Get "http://10.132.0.129:15021/healthz/ready": dial tcp 10.132.0.129:15021: connect: connection refused 30m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Startup probe failed: Get "http://10.133.0.169:15021/healthz/ready": dial tcp 10.133.0.169:15021: connect: connection refused 30m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-f43174-56964f47-rqms4 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-shared-b-w5-f43174-565fcdf8cf-ws4zx Stopping container payload-processing 30m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-fhqht Stopping container istio-proxy 30m Normal Killing pod/e2e-worker-w1-4fe3a4-openshift-default-5d7894fd77-mr56h Stopping container istio-proxy 30m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-f43174-openshift-default Scaled down replica set e2e-shared-b-w5-f43174-openshift-default-5f7c77757 from 4 to 3 30m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-f43174-openshift-default-5f7c77757 Deleted pod: e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-f43174-openshift-default New size: 3; reason: All metrics below target 30m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-4bwln Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal Killing pod/e2e-worker-w4-59357a-openshift-default-5b84865d9c-gn5jk Stopping container istio-proxy 30m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Stopping container istio-proxy 30m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Stopping container istio-proxy 30m Normal Killing pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Stopping container payload-processing 30m Normal Killing pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Stopping container istio-proxy 30m Normal Killing pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Stopping container payload-pre-processing 30m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-ksjff Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-bjl9w Readiness probe error: HTTP probe failed with statuscode: 503... 30m Warning Unhealthy pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-shared-b-w5-f43174-openshift-default-5f7c77757-d8vff Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Stopping container payload-pre-processing 29m Normal ScalingReplicaSet deployment/e2e-worker-w3-f7821a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 from 2 to 1 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-ac1273-openshift-default New size: 1; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w0-ac1273-openshift-default-79699df87 Deleted pod: e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-f7821a-openshift-default New size: 1; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w0-ac1273-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-ac1273-openshift-default-79699df87 from 2 to 1 29m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-xjtq4 Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7 Deleted pod: e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-9qxgj 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-ac1273-787cd546f5-9gg7q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-processing-e2e-worker-w0-ac1273-74fc6446bf-ccs66 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Stopping container payload-processing 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-ef6d93-6b75448499-hll95 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-processing-e2e-worker-w5-ef6d93-847d6594c9-xdh7n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-x5xsx Stopping container istio-proxy 29m Warning Unhealthy pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning ProbeError pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/e2e-worker-w0-ac1273-openshift-default-79699df87-5d4lt Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-worker-w5-ef6d93-openshift-default-58df979549-d6k42 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-a1685e-cfbd66747-h6jcv Readiness probe failed: timeout: failed to connect service "10.133.0.38:50052" within 1s: context deadline exceeded 29m Warning Unhealthy pod/payload-processing-e2e-worker-w6-a1685e-7f8694fc87-ssvs7 Readiness probe failed: timeout: failed to connect service "10.133.0.39:50052" within 1s: context deadline exceeded 28m Normal ScalingReplicaSet deployment/e2e-worker-w6-a1685e-openshift-default (combined from similar events): Scaled up replica set e2e-worker-w6-a1685e-openshift-default-684b58587c from 2 to 3 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-a1685e-openshift-default (combined from similar events): New size: 3; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-worker-w6-a1685e-openshift-default-684b58587c Created pod: e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w 28m Normal Pulled pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w Container created 28m Normal Started pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w Container started 28m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-cqlbk Stopping container istio-proxy 28m Normal AddedInterface pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w Add eth0 [10.134.0.217/23] from ovn-kubernetes 28m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-8ncqx Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w6-a1685e-openshift-default-684b58587c-v649w Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-ndttp Stopping container discovery 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-ndttp 25m Normal Killing pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Stopping container payload-pre-processing 24m Normal Killing pod/e2e-inf-a-05db63-openshift-default-8cdbcbbdc-2r9c9 Stopping container istio-proxy 24m Warning Unhealthy pod/payload-processing-e2e-inf-a-05db63-6ddf9bc6f7-llqzz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-05db63-68cc6dc67f-9b9fj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Stopping container payload-processing 24m Warning Unhealthy pod/payload-processing-e2e-inf-b-66db1c-77f86d9bbc-7rsl6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/e2e-inf-b-66db1c-openshift-default-798cbbdd45-5ds9w Stopping container istio-proxy 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-66db1c-d95868c6c-npsqs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-x2fs8 Stopping container istio-proxy 23m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-m7bsq Stopping container istio-proxy 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-qr6gj Stopping container istio-proxy 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-ghk9c Stopping container istio-proxy 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Stopping container istio-proxy 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-mqgxj Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-cnft9 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-cpsqt Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-fcb75c45f Deleted pod: maas-default-gateway-openshift-default-fcb75c45f-cpsqt 23m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-cnft9 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/payload-pre-processing-e2e-worker-w3-f7821a-7c94f8c899-7xkrb Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-worker-w3-f7821a-7bd5cd9755-dl8q2 Stopping container payload-processing 23m Normal Killing pod/maas-default-gateway-openshift-default-fcb75c45f-ldrj5 Stopping container istio-proxy 23m Normal Killing pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Stopping container istio-proxy 23m Warning ProbeError pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-worker-w3-f7821a-openshift-default-b4ddf4df7-frjdl Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Add eth0 [10.133.0.175/23] from ovn-kubernetes 22m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Container created 22m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Container started 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Startup probe failed: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Startup probe error: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused... 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Add eth0 [10.133.0.177/23] from ovn-kubernetes 22m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Container created 22m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Add eth0 [10.134.0.226/23] from ovn-kubernetes 22m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Container started 22m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Container created 22m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Container started 22m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 5; reason: cpu resource utilization (percentage of request) above target 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Startup probe failed: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Startup probe error: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused... 22m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 22m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-fcb75c45f from 5 to 10 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 21m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Add eth0 [10.132.0.132/23] from ovn-kubernetes 21m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Container started 21m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Container created 21m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Container created 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Add eth0 [10.132.0.133/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Container created 21m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Container created 21m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Container started 21m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Container started 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Add eth0 [10.134.0.227/23] from ovn-kubernetes 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Add eth0 [10.133.0.181/23] from ovn-kubernetes 21m Normal Pulled pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Add eth0 [10.133.0.180/23] from ovn-kubernetes 21m Normal Created pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Container created 21m Normal Started pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Container started 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Startup probe failed: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": dial tcp 10.133.0.180:15021: connect: connection refused 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Startup probe error: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Startup probe failed: Get "http://10.134.0.227:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Startup probe error: Get "http://10.134.0.227:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Startup probe error: Get "http://10.133.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Startup probe failed: Get "http://10.133.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Readiness probe error: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-m4gjr Readiness probe failed: Get "http://10.132.0.132:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal AddedLabel gateway/e2e-worker-single-main-f191a7 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-f191a7 15m Normal SuccessfulCreate replicaset/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdf Created pod: e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt 15m Normal ScalingReplicaSet deployment/e2e-worker-single-main-f191a7-openshift-default Scaled up replica set e2e-worker-single-main-f191a7-openshift-default-58b78bdcdf from 0 to 1 15m Normal AddedInterface pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Add eth0 [10.133.0.195/23] from ovn-kubernetes 15m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-f191a7 Scaled up replica set payload-pre-processing-e2e-worker-single-main-f191a7-55d4d7496c from 0 to 1 15m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-f191a7 Scaled up replica set payload-processing-e2e-worker-single-main-f191a7-66c4865c8f from 0 to 1 15m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-f191a7-66c4865c8f Created pod: payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch 15m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-f191a7-66c4865c8f Error creating: pods "payload-processing-e2e-worker-single-main-f191a7-66c4865c8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-f191a7: serviceaccount "payload-processing-e2e-worker-single-main-f191a7" not found 15m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Add eth0 [10.133.0.196/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d7496c Created pod: payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb 15m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d7496c Error creating: pods "payload-pre-processing-e2e-worker-single-main-f191a7-55d4d7496c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-f191a7: serviceaccount "payload-processing-e2e-worker-single-main-f191a7" not found 15m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Created pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Container created 15m Normal Started pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Container started 15m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Add eth0 [10.133.0.197/23] from ovn-kubernetes 15m Normal Pulled pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 15m Normal Created pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Container created 15m Normal Started pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Container started 15m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-f191a7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-f191a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 15m Normal Started pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Container started 15m Normal Created pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Container created 15m Normal Pulled pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 14m Warning ProbeError pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 14m Warning Unhealthy pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 14m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-f191a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 14m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-f191a7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 13m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Stopping container payload-pre-processing 13m Normal Killing pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Stopping container payload-processing 13m Normal Killing pod/e2e-worker-single-main-f191a7-openshift-default-58b78bdcdftmvzt Stopping container istio-proxy 13m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-f191a7-55d4d6pcxb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-f191a7-66c4865c8zsdch Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 11m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Readiness probe error: Get "http://10.133.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-dx8zr Readiness probe failed: Get "http://10.133.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Normal AddedInterface pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Add eth0 [10.133.0.205/23] from ovn-kubernetes 10m Normal AddedLabel gateway/e2e-worker-single-main-a7bea2 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-a7bea2 10m Normal ScalingReplicaSet deployment/e2e-worker-single-main-a7bea2-openshift-default Scaled up replica set e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c from 0 to 1 10m Normal SuccessfulCreate replicaset/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c Created pod: e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk 10m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-a7bea2 Scaled up replica set payload-pre-processing-e2e-worker-single-main-a7bea2-dc887958c from 0 to 1 10m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-a7bea2-dc887958c Created pod: payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t 10m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-a7bea2-dc887958c Error creating: pods "payload-pre-processing-e2e-worker-single-main-a7bea2-dc887958c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a7bea2: serviceaccount "payload-processing-e2e-worker-single-main-a7bea2" not found 10m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-a7bea2-658b44bc Error creating: pods "payload-processing-e2e-worker-single-main-a7bea2-658b44bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a7bea2: serviceaccount "payload-processing-e2e-worker-single-main-a7bea2" not found 10m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-a7bea2-658b44bc Created pod: payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf 10m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-a7bea2 Scaled up replica set payload-processing-e2e-worker-single-main-a7bea2-658b44bc from 0 to 1 10m Normal Started pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Container started 10m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Add eth0 [10.133.0.206/23] from ovn-kubernetes 10m Normal Created pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Container created 10m Normal Pulled pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Add eth0 [10.133.0.207/23] from ovn-kubernetes 10m Normal Started pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Container started 10m Normal Created pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Container created 10m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 10m Normal Pulled pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 10m Normal Started pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Container started 10m Normal Created pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Container created 10m Warning Unhealthy pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Startup probe failed: Get "http://10.133.0.205:15021/healthz/ready": dial tcp 10.133.0.205:15021: connect: connection refused 10m Warning ProbeError pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Startup probe error: Get "http://10.133.0.205:15021/healthz/ready": dial tcp 10.133.0.205:15021: connect: connection refused... 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a7bea2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a7bea2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 9m50s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a7bea2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m50s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a7bea2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 9m8s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Stopping container payload-pre-processing 9m8s Normal Killing pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Stopping container payload-processing 8m55s Normal Killing pod/e2e-worker-single-main-a7bea2-openshift-default-58b6cb87c-nf7lk Stopping container istio-proxy 8m49s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-a7bea2-dc8877xs6t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m49s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-a7bea2-658b44bc-vcltf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Readiness probe error: Get "http://10.132.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 8m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Readiness probe failed: Get "http://10.134.0.227:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-sr5pb Readiness probe failed: Get "http://10.132.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 8m Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-r5grr Readiness probe error: Get "http://10.134.0.227:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m59s Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Readiness probe failed: Get "http://10.133.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m59s Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-x6tl6 Readiness probe error: Get "http://10.133.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m33s Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Readiness probe failed: Get "http://10.133.0.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m33s Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Readiness probe failed: Get "http://10.134.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 7m33s Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-8965v Readiness probe error: Get "http://10.133.0.175:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 7m33s Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-5tkrq Readiness probe error: Get "http://10.134.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m59s Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Readiness probe error: Get "http://10.133.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m59s Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-s8bll Readiness probe failed: Get "http://10.133.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m23s Normal AddedLabel gateway/e2e-ait-c6ac3112 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-c6ac3112 6m22s Normal SuccessfulCreate replicaset/e2e-ait-c6ac3112-openshift-default-6875446d77 Created pod: e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q 6m22s Normal ScalingReplicaSet deployment/e2e-ait-c6ac3112-openshift-default Scaled up replica set e2e-ait-c6ac3112-openshift-default-6875446d77 from 0 to 1 6m22s Normal AddedInterface pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Add eth0 [10.133.0.217/23] from ovn-kubernetes 6m15s Warning FailedCreate replicaset/payload-processing-e2e-ait-c6ac3112-f6678b99f Error creating: pods "payload-processing-e2e-ait-c6ac3112-f6678b99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c6ac3112: serviceaccount "payload-processing-e2e-ait-c6ac3112" not found 6m15s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5 Error creating: pods "payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-c6ac3112: serviceaccount "payload-processing-e2e-ait-c6ac3112" not found 6m15s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5 Created pod: payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c 6m15s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-c6ac3112 Scaled up replica set payload-pre-processing-e2e-ait-c6ac3112-55976f95c5 from 0 to 1 6m15s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-c6ac3112-f6678b99f Created pod: payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf 6m15s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-c6ac3112 Scaled up replica set payload-processing-e2e-ait-c6ac3112-f6678b99f from 0 to 1 6m14s Normal Created pod/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf Container created 6m14s Normal AddedInterface pod/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf Add eth0 [10.133.0.219/23] from ovn-kubernetes 6m14s Normal Pulled pod/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m14s Normal AddedInterface pod/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c Add eth0 [10.133.0.218/23] from ovn-kubernetes 6m14s Normal Started pod/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c Container started 6m14s Normal Created pod/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c Container created 6m14s Normal Pulled pod/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 6m14s Normal Started pod/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf Container started 5m55s Normal Started pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Container started 5m55s Normal Created pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Container created 5m55s Normal Pulled pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m49s Warning Unhealthy pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 5m49s Warning ProbeError pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 5m31s Normal Killing pod/payload-pre-processing-e2e-ait-c6ac3112-55976f95c5-hk97c Stopping container payload-pre-processing 5m31s Normal Killing pod/payload-processing-e2e-ait-c6ac3112-f6678b99f-wdvpf Stopping container payload-processing 5m22s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-c6ac3112-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m22s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-c6ac3112-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 5m21s Normal Killing pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Stopping container istio-proxy 5m20s Normal ScalingReplicaSet deployment/e2e-authcfg-http-713882-openshift-default Scaled up replica set e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7 from 0 to 1 5m20s Normal AddedLabel gateway/e2e-authcfg-http-713882 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-713882 5m19s Normal AddedInterface pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Add eth0 [10.133.0.221/23] from ovn-kubernetes 5m19s Normal SuccessfulCreate replicaset/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7 Created pod: e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd 5m18s Warning ProbeError pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Readiness probe error: HTTP probe failed with statuscode: 503... 5m18s Warning Unhealthy pod/e2e-ait-c6ac3112-openshift-default-6875446d77-kc77q Readiness probe failed: HTTP probe failed with statuscode: 503 5m12s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-713882-7679498475 Created pod: payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 5m12s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-713882 Scaled up replica set payload-pre-processing-e2e-authcfg-http-713882-7679498475 from 0 to 1 5m12s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-713882 Scaled up replica set payload-processing-e2e-authcfg-http-713882-65f8b499cc from 0 to 1 5m12s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-713882-65f8b499cc Created pod: payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 5m12s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-713882-7679498475 Error creating: pods "payload-pre-processing-e2e-authcfg-http-713882-7679498475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-713882: serviceaccount "payload-processing-e2e-authcfg-http-713882" not found 5m12s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-713882-65f8b499cc Error creating: pods "payload-processing-e2e-authcfg-http-713882-65f8b499cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-713882: serviceaccount "payload-processing-e2e-authcfg-http-713882" not found 5m11s Normal Started pod/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 Container started 5m11s Normal Created pod/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 Container created 5m11s Normal Created pod/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 Container created 5m11s Normal Started pod/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 Container started 5m11s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 Add eth0 [10.133.0.222/23] from ovn-kubernetes 5m11s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 Add eth0 [10.133.0.223/23] from ovn-kubernetes 5m11s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m11s Normal Pulled pod/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m52s Normal Pulled pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m52s Normal Created pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Container created 4m52s Normal Started pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Container started 4m49s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-713882-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m49s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-713882-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m47s Warning ProbeError pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 4m47s Warning Unhealthy pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 4m19s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-713882-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m19s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-713882-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m19s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-713882-7679498475-94m92 Stopping container payload-pre-processing 3m19s Normal Killing pod/payload-processing-e2e-authcfg-http-713882-65f8b499cc-8xsr2 Stopping container payload-processing 3m Normal Killing pod/e2e-authcfg-http-713882-openshift-default-6bc7bd5ff7-kcbmd Stopping container istio-proxy 2m39s Warning ProbeError pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Readiness probe error: Get "http://10.132.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m39s Warning Unhealthy pod/maas-default-gateway-openshift-default-fcb75c45f-jdhp6 Readiness probe failed: Get "http://10.132.0.34:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)