=== events in openshift-ingress (2026-10-30T01:39:04Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 to ip-10-0-128-7.ec2.internal 5m34s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx to ip-10-0-135-56.ec2.internal 9m20s Normal Scheduled pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Successfully assigned openshift-ingress/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn to ip-10-0-135-56.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm to ip-10-0-135-56.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 to ip-10-0-131-130.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv to ip-10-0-131-130.ec2.internal 5m39s Normal Scheduled pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Successfully assigned openshift-ingress/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr to ip-10-0-131-130.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl to ip-10-0-128-7.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch to ip-10-0-128-7.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c to ip-10-0-131-130.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp to ip-10-0-135-56.ec2.internal 49m Normal Scheduled pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Successfully assigned openshift-ingress/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 to ip-10-0-135-56.ec2.internal 31m Normal Scheduled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Successfully assigned openshift-ingress/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 to ip-10-0-128-7.ec2.internal 9m15s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd to ip-10-0-135-56.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm to ip-10-0-131-130.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Successfully assigned openshift-ingress/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 to ip-10-0-128-7.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n to ip-10-0-131-130.ec2.internal 31m Normal Scheduled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Successfully assigned openshift-ingress/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 to ip-10-0-131-130.ec2.internal 31m Normal Scheduled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Successfully assigned openshift-ingress/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l to ip-10-0-135-56.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd to ip-10-0-131-130.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Successfully assigned openshift-ingress/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr to ip-10-0-135-56.ec2.internal 27m Normal Scheduled pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Successfully assigned openshift-ingress/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt to ip-10-0-128-7.ec2.internal 48m Normal Scheduled pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Successfully assigned openshift-ingress/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh to ip-10-0-135-56.ec2.internal 30m Normal Scheduled pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Successfully assigned openshift-ingress/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x to ip-10-0-135-56.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Successfully assigned openshift-ingress/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b to ip-10-0-128-7.ec2.internal 56m Normal Scheduled pod/payload-processing-5f78687d5b-l4gst Successfully assigned openshift-ingress/payload-processing-5f78687d5b-l4gst to ip-10-0-135-56.ec2.internal 26m Normal Scheduled pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Successfully assigned openshift-ingress/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz to ip-10-0-135-56.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 Successfully assigned openshift-ingress/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 to ip-10-0-135-56.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Successfully assigned openshift-ingress/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb to ip-10-0-131-130.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt to ip-10-0-128-7.ec2.internal 52m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm to ip-10-0-128-7.ec2.internal 47m Normal Scheduled pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Successfully assigned openshift-ingress/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 Successfully assigned openshift-ingress/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 to ip-10-0-135-56.ec2.internal 46m Normal Scheduled pod/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 to ip-10-0-135-56.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j to ip-10-0-131-130.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Successfully assigned openshift-ingress/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v to ip-10-0-128-7.ec2.internal 46m Normal Scheduled pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Successfully assigned openshift-ingress/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 to ip-10-0-131-130.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz to ip-10-0-128-7.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 to ip-10-0-131-130.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n to ip-10-0-131-130.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d to ip-10-0-131-130.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl to ip-10-0-131-130.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw to ip-10-0-131-130.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp to ip-10-0-135-56.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Successfully assigned openshift-ingress/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Successfully assigned openshift-ingress/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd to ip-10-0-131-130.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Successfully assigned openshift-ingress/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw to ip-10-0-128-7.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Successfully assigned openshift-ingress/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Successfully assigned openshift-ingress/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 to ip-10-0-131-130.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h to ip-10-0-135-56.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv to ip-10-0-131-130.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc to ip-10-0-131-130.ec2.internal 46m Normal Scheduled pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Successfully assigned openshift-ingress/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s to ip-10-0-131-130.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 to ip-10-0-128-7.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b to ip-10-0-131-130.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x to ip-10-0-131-130.ec2.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 to ip-10-0-135-56.ec2.internal 48m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x to ip-10-0-135-56.ec2.internal 36m Normal Scheduled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Successfully assigned openshift-ingress/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 to ip-10-0-128-7.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp to ip-10-0-128-7.ec2.internal 34m Normal Scheduled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Successfully assigned openshift-ingress/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc to ip-10-0-135-56.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx to ip-10-0-135-56.ec2.internal 20m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b to ip-10-0-131-130.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Successfully assigned openshift-ingress/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 to ip-10-0-135-56.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Successfully assigned openshift-ingress/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs to ip-10-0-128-7.ec2.internal 48m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw to ip-10-0-131-130.ec2.internal 54m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h to ip-10-0-135-56.ec2.internal 43m Normal Scheduled pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Successfully assigned openshift-ingress/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc to ip-10-0-131-130.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 to ip-10-0-131-130.ec2.internal 52m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 to ip-10-0-128-7.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Successfully assigned openshift-ingress/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp to ip-10-0-135-56.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn to ip-10-0-131-130.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Successfully assigned openshift-ingress/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc to ip-10-0-135-56.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb to ip-10-0-128-7.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Successfully assigned openshift-ingress/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq to ip-10-0-128-7.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r Successfully assigned openshift-ingress/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r to ip-10-0-135-56.ec2.internal 20m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 to ip-10-0-135-56.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 to ip-10-0-131-130.ec2.internal 56m Normal Scheduled pod/payload-pre-processing-7b98bb678c-qcfkv Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-qcfkv to ip-10-0-131-130.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz to ip-10-0-128-7.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 to ip-10-0-128-7.ec2.internal 44m Normal Scheduled pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Successfully assigned openshift-ingress/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc to ip-10-0-128-7.ec2.internal 55m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt to ip-10-0-131-130.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Successfully assigned openshift-ingress/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp to ip-10-0-128-7.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl to ip-10-0-131-130.ec2.internal 52m Normal Scheduled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Successfully assigned openshift-ingress/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h to ip-10-0-131-130.ec2.internal 32m Normal Scheduled pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Successfully assigned openshift-ingress/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc to ip-10-0-131-130.ec2.internal 30m Normal Scheduled pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Successfully assigned openshift-ingress/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 to ip-10-0-135-56.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh to ip-10-0-131-130.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l to ip-10-0-131-130.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx to ip-10-0-131-130.ec2.internal 27m Normal Scheduled pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Successfully assigned openshift-ingress/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 to ip-10-0-135-56.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f to ip-10-0-131-130.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf to ip-10-0-128-7.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx to ip-10-0-128-7.ec2.internal 53m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz to ip-10-0-135-56.ec2.internal 42m Normal Scheduled pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Successfully assigned openshift-ingress/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Successfully assigned openshift-ingress/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt to ip-10-0-128-7.ec2.internal 53m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb to ip-10-0-131-130.ec2.internal 36m Normal Scheduled pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Successfully assigned openshift-ingress/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk to ip-10-0-131-130.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z to ip-10-0-128-7.ec2.internal 51m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k to ip-10-0-131-130.ec2.internal 54m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv to ip-10-0-128-7.ec2.internal 31m Normal Scheduled pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Successfully assigned openshift-ingress/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg to ip-10-0-128-7.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Successfully assigned openshift-ingress/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h to ip-10-0-131-130.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 to ip-10-0-128-7.ec2.internal 28m Normal Scheduled pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Successfully assigned openshift-ingress/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 to ip-10-0-135-56.ec2.internal 51m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh to ip-10-0-131-130.ec2.internal 45m Normal Scheduled pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Successfully assigned openshift-ingress/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr to ip-10-0-128-7.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn to ip-10-0-131-130.ec2.internal 52m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw to ip-10-0-128-7.ec2.internal 29m Normal Scheduled pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Successfully assigned openshift-ingress/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs to ip-10-0-131-130.ec2.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 to ip-10-0-135-56.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 to ip-10-0-131-130.ec2.internal 52m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd to ip-10-0-135-56.ec2.internal 51m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Successfully assigned openshift-ingress/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv to ip-10-0-135-56.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Successfully assigned openshift-ingress/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k to ip-10-0-128-7.ec2.internal 33m Normal Scheduled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Successfully assigned openshift-ingress/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp to ip-10-0-131-130.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Successfully assigned openshift-ingress/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh to ip-10-0-135-56.ec2.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-ws84g to ip-10-0-131-130.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg to ip-10-0-128-7.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-srsm4 to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Successfully assigned openshift-ingress/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j to ip-10-0-131-130.ec2.internal 37m Normal Scheduled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Successfully assigned openshift-ingress/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 to ip-10-0-128-7.ec2.internal 54m Normal Scheduled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Successfully assigned openshift-ingress/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj to ip-10-0-128-7.ec2.internal 42m Normal Scheduled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Successfully assigned openshift-ingress/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Successfully assigned openshift-ingress/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 to ip-10-0-128-7.ec2.internal 56m Normal Scheduled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Successfully assigned openshift-ingress/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b to ip-10-0-131-130.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Successfully assigned openshift-ingress/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Successfully assigned openshift-ingress/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Successfully assigned openshift-ingress/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 to ip-10-0-135-56.ec2.internal 31m Normal Scheduled pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Successfully assigned openshift-ingress/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r to ip-10-0-128-7.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Successfully assigned openshift-ingress/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Successfully assigned openshift-ingress/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 to ip-10-0-135-56.ec2.internal 46m Normal Scheduled pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Successfully assigned openshift-ingress/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg to ip-10-0-128-7.ec2.internal 46m Normal Scheduled pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Successfully assigned openshift-ingress/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Successfully assigned openshift-ingress/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r to ip-10-0-131-130.ec2.internal 45m Normal Scheduled pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Successfully assigned openshift-ingress/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Successfully assigned openshift-ingress/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 to ip-10-0-135-56.ec2.internal 49m Normal Scheduled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Successfully assigned openshift-ingress/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Successfully assigned openshift-ingress/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 to ip-10-0-128-7.ec2.internal 49m Normal Scheduled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Successfully assigned openshift-ingress/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf to ip-10-0-131-130.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k to ip-10-0-131-130.ec2.internal 42m Normal Scheduled pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Successfully assigned openshift-ingress/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 to ip-10-0-131-130.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h to ip-10-0-131-130.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq to ip-10-0-131-130.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 to ip-10-0-131-130.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp to ip-10-0-131-130.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj to ip-10-0-128-7.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj to ip-10-0-131-130.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-s47sp to ip-10-0-131-130.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr to ip-10-0-131-130.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct to ip-10-0-131-130.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 to ip-10-0-128-7.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Successfully assigned openshift-ingress/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g to ip-10-0-131-130.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx to ip-10-0-135-56.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Successfully assigned openshift-ingress/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 to ip-10-0-128-7.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp Successfully assigned openshift-ingress/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp to ip-10-0-131-130.ec2.internal 62m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw to ip-10-0-135-56.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq to ip-10-0-131-130.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch to ip-10-0-131-130.ec2.internal 16m Normal Scheduled pod/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx to ip-10-0-131-130.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 to ip-10-0-131-130.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c to ip-10-0-135-56.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh to ip-10-0-131-130.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Successfully assigned openshift-ingress/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl to ip-10-0-131-130.ec2.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-lxc86 to ip-10-0-131-130.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb to ip-10-0-131-130.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-lvppb to ip-10-0-131-130.ec2.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-kxdph to ip-10-0-131-130.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb to ip-10-0-131-130.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-kgxxw to ip-10-0-131-130.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt to ip-10-0-128-7.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn Successfully assigned openshift-ingress/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn to ip-10-0-135-56.ec2.internal 25m Normal Scheduled pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Successfully assigned openshift-ingress/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-kfpcv to ip-10-0-135-56.ec2.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-jwx9q to ip-10-0-128-7.ec2.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 to ip-10-0-131-130.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj to ip-10-0-131-130.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 to ip-10-0-128-7.ec2.internal 44m Normal Scheduled pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Successfully assigned openshift-ingress/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 to ip-10-0-128-7.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 Successfully assigned openshift-ingress/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 to ip-10-0-131-130.ec2.internal 52m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 to ip-10-0-135-56.ec2.internal 22m Normal Scheduled pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Successfully assigned openshift-ingress/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 to ip-10-0-128-7.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 Successfully assigned openshift-ingress/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 to ip-10-0-135-56.ec2.internal 53m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq to ip-10-0-128-7.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Successfully assigned openshift-ingress/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-gfvgn to ip-10-0-131-130.ec2.internal 27m Normal Scheduled pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Successfully assigned openshift-ingress/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd to ip-10-0-128-7.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Successfully assigned openshift-ingress/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw to ip-10-0-131-130.ec2.internal 52m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl to ip-10-0-135-56.ec2.internal 51m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w to ip-10-0-135-56.ec2.internal 18m Normal Scheduled pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Successfully assigned openshift-ingress/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj to ip-10-0-128-7.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-fwrpg to ip-10-0-128-7.ec2.internal 46m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-fv487 to ip-10-0-131-130.ec2.internal 29m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp to ip-10-0-131-130.ec2.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq to ip-10-0-135-56.ec2.internal 19m Normal Scheduled pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Successfully assigned openshift-ingress/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q to ip-10-0-128-7.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg to ip-10-0-135-56.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p to ip-10-0-131-130.ec2.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-cngvb to ip-10-0-135-56.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn to ip-10-0-131-130.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Successfully assigned openshift-ingress/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x to ip-10-0-128-7.ec2.internal 30m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 to ip-10-0-131-130.ec2.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j to ip-10-0-131-130.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-ccb2j to ip-10-0-131-130.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 to ip-10-0-135-56.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 to ip-10-0-135-56.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h to ip-10-0-131-130.ec2.internal 41m Normal Scheduled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Successfully assigned openshift-ingress/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj to ip-10-0-131-130.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh to ip-10-0-131-130.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx to ip-10-0-128-7.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 to ip-10-0-128-7.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 to ip-10-0-131-130.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs to ip-10-0-131-130.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-97m56 to ip-10-0-135-56.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r to ip-10-0-128-7.ec2.internal 21m Normal Scheduled pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Successfully assigned openshift-ingress/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj to ip-10-0-128-7.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Successfully assigned openshift-ingress/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw to ip-10-0-135-56.ec2.internal 52m Normal Scheduled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Successfully assigned openshift-ingress/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 to ip-10-0-128-7.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v to ip-10-0-131-130.ec2.internal 23m Normal Scheduled pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Successfully assigned openshift-ingress/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb to ip-10-0-131-130.ec2.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-4nbkc to ip-10-0-128-7.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk to ip-10-0-131-130.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 to ip-10-0-131-130.ec2.internal 40m Normal Scheduled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Successfully assigned openshift-ingress/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n to ip-10-0-131-130.ec2.internal 16m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg to ip-10-0-131-130.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn to ip-10-0-131-130.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 to ip-10-0-131-130.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 to ip-10-0-131-130.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n to ip-10-0-135-56.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-2jzrh to ip-10-0-128-7.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv to ip-10-0-131-130.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd to ip-10-0-128-7.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn to ip-10-0-135-56.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb to ip-10-0-135-56.ec2.internal 21m Normal Scheduled pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Successfully assigned openshift-ingress/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 to ip-10-0-128-7.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7d9897b54c-26zqk to ip-10-0-135-56.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t to ip-10-0-128-7.ec2.internal 47m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z to ip-10-0-128-7.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf to ip-10-0-135-56.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Successfully assigned openshift-ingress/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q to ip-10-0-128-7.ec2.internal 30m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 to ip-10-0-131-130.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n to ip-10-0-131-130.ec2.internal 28m Normal Scheduled pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Successfully assigned openshift-ingress/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh to ip-10-0-131-130.ec2.internal 42m Normal Scheduled pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Successfully assigned openshift-ingress/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 to ip-10-0-131-130.ec2.internal 62m Normal Scheduled pod/kube-auth-proxy-b7df967f4-w85dq Successfully assigned openshift-ingress/kube-auth-proxy-b7df967f4-w85dq to ip-10-0-131-130.ec2.internal 9m14s Normal Scheduled pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 to ip-10-0-135-56.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Successfully assigned openshift-ingress/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq to ip-10-0-128-7.ec2.internal 52m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg to ip-10-0-128-7.ec2.internal 5m34s Normal Scheduled pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx to ip-10-0-131-130.ec2.internal 37m Normal Scheduled pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Successfully assigned openshift-ingress/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc to ip-10-0-128-7.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Successfully assigned openshift-ingress/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 to ip-10-0-128-7.ec2.internal 52m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 to ip-10-0-131-130.ec2.internal 40m Normal Scheduled pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Successfully assigned openshift-ingress/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc to ip-10-0-131-130.ec2.internal 62m Normal Scheduled pod/kube-auth-proxy-b7df967f4-5hvrq Successfully assigned openshift-ingress/kube-auth-proxy-b7df967f4-5hvrq to ip-10-0-135-56.ec2.internal 20m Normal Scheduled pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Successfully assigned openshift-ingress/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx to ip-10-0-128-7.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm to ip-10-0-131-130.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Successfully assigned openshift-ingress/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 to ip-10-0-135-56.ec2.internal 29m Normal Scheduled pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Successfully assigned openshift-ingress/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 to ip-10-0-128-7.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf to ip-10-0-135-56.ec2.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 to ip-10-0-131-130.ec2.internal 30m Normal Scheduled pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Successfully assigned openshift-ingress/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 to ip-10-0-131-130.ec2.internal 70m Warning FailedScheduling pod/router-default-85f7b79cf7-h2tpp no nodes available to schedule pods 67m Warning FailedScheduling pod/router-default-85f7b79cf7-h2tpp no nodes available to schedule pods 34m Normal Scheduled pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Successfully assigned openshift-ingress/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 to ip-10-0-131-130.ec2.internal 67m Normal Scheduled pod/router-default-85f7b79cf7-h2tpp Successfully assigned openshift-ingress/router-default-85f7b79cf7-h2tpp to ip-10-0-128-7.ec2.internal 52m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg to ip-10-0-135-56.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d to ip-10-0-131-130.ec2.internal 54m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct to ip-10-0-128-7.ec2.internal 38m Normal Scheduled pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Successfully assigned openshift-ingress/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws to ip-10-0-131-130.ec2.internal 55m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 to ip-10-0-131-130.ec2.internal 41m Normal Scheduled pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Successfully assigned openshift-ingress/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw to ip-10-0-128-7.ec2.internal 34m Normal Scheduled pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Successfully assigned openshift-ingress/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn to ip-10-0-135-56.ec2.internal 53m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb to ip-10-0-131-130.ec2.internal 41m Normal Scheduled pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Successfully assigned openshift-ingress/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw to ip-10-0-131-130.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm to ip-10-0-135-56.ec2.internal 25m Normal Scheduled pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Successfully assigned openshift-ingress/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 to ip-10-0-128-7.ec2.internal 53m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 to ip-10-0-135-56.ec2.internal 61m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 to ip-10-0-135-56.ec2.internal 62m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-d695g Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-d695g to ip-10-0-135-56.ec2.internal 38m Normal Scheduled pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Successfully assigned openshift-ingress/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq to ip-10-0-128-7.ec2.internal 44m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn to ip-10-0-135-56.ec2.internal 47m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 to ip-10-0-128-7.ec2.internal 54m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc to ip-10-0-128-7.ec2.internal 43m Normal Scheduled pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Successfully assigned openshift-ingress/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 to ip-10-0-131-130.ec2.internal 47m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq to ip-10-0-131-130.ec2.internal 40m Normal Scheduled pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Successfully assigned openshift-ingress/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz to ip-10-0-128-7.ec2.internal 44m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp to ip-10-0-131-130.ec2.internal 49m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-j5x7c to ip-10-0-128-7.ec2.internal 40m Normal Scheduled pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Successfully assigned openshift-ingress/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 to ip-10-0-128-7.ec2.internal 43m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-5gbdq to ip-10-0-131-130.ec2.internal 44m Normal Scheduled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Successfully assigned openshift-ingress/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 to ip-10-0-128-7.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv to ip-10-0-128-7.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 to ip-10-0-131-130.ec2.internal 34m Normal Scheduled pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Successfully assigned openshift-ingress/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d to ip-10-0-135-56.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf to ip-10-0-131-130.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Successfully assigned openshift-ingress/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl to ip-10-0-131-130.ec2.internal 53m Normal Scheduled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Successfully assigned openshift-ingress/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx to ip-10-0-135-56.ec2.internal 71m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-85f7b79cf7 from 0 to 1 71m Warning FailedCreate replicaset/router-default-85f7b79cf7 Error creating: pods "router-default-85f7b79cf7-" is forbidden: error fetching namespace "openshift-ingress": unable to find annotation openshift.io/sa.scc.uid-range 70m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 70m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 70m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 70m Normal SuccessfulCreate replicaset/router-default-85f7b79cf7 Created pod: router-default-85f7b79cf7-h2tpp 67m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 67m Warning FailedMount pod/router-default-85f7b79cf7-h2tpp MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 67m Warning FailedMount pod/router-default-85f7b79cf7-h2tpp MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 67m Warning FailedMount pod/router-default-85f7b79cf7-h2tpp MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 67m Normal Started pod/router-default-85f7b79cf7-h2tpp Container started 67m Normal AddedInterface pod/router-default-85f7b79cf7-h2tpp Add eth0 [10.132.0.9/23] from ovn-kubernetes 67m Normal Created pod/router-default-85f7b79cf7-h2tpp Container created 67m Normal Pulled pod/router-default-85f7b79cf7-h2tpp Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 62m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-b7df967f4 from 0 to 2 62m Normal SuccessfulCreate replicaset/kube-auth-proxy-b7df967f4 Created pod: kube-auth-proxy-b7df967f4-5hvrq 62m Normal SuccessfulCreate replicaset/kube-auth-proxy-b7df967f4 Created pod: kube-auth-proxy-b7df967f4-w85dq 62m Normal AddedInterface pod/kube-auth-proxy-b7df967f4-5hvrq Add eth0 [10.133.0.26/23] from ovn-kubernetes 62m Normal Pulling pod/kube-auth-proxy-b7df967f4-w85dq Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 62m Normal AddedInterface pod/kube-auth-proxy-b7df967f4-w85dq Add eth0 [10.134.0.22/23] from ovn-kubernetes 62m Normal Pulling pod/kube-auth-proxy-b7df967f4-5hvrq Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 62m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-d695g 62m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 62m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-d695g Add eth0 [10.133.0.27/23] from ovn-kubernetes 62m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-d695g Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 62m Normal Created pod/kube-auth-proxy-b7df967f4-w85dq Container created 62m Normal Pulled pod/kube-auth-proxy-b7df967f4-5hvrq Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.514s (2.514s including waiting). Image size: 188904677 bytes. 62m Normal Started pod/kube-auth-proxy-b7df967f4-w85dq Container started 62m Normal Created pod/kube-auth-proxy-b7df967f4-5hvrq Container created 62m Normal Pulled pod/kube-auth-proxy-b7df967f4-w85dq Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.491s (2.491s including waiting). Image size: 188904677 bytes. 62m Normal Started pod/kube-auth-proxy-b7df967f4-5hvrq Container started 62m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-d695g Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.648s (2.648s including waiting). Image size: 185476477 bytes. 62m Normal Created pod/istiod-openshift-gateway-7876bfcb74-d695g Container created 62m Normal Started pod/istiod-openshift-gateway-7876bfcb74-d695g Container started 62m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-858f55f4d9 from 0 to 1 62m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-858f55f4d9 Created pod: data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw 62m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw Add eth0 [10.133.0.28/23] from ovn-kubernetes 62m Normal Pulling pod/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 62m Normal Pulled pod/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.813s (2.813s including waiting). Image size: 183555333 bytes. 62m Normal Created pod/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw Container created 62m Normal Started pod/data-science-gateway-data-science-gateway-class-858f55f4d9kq5rw Container started 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-k5nr4 61m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Container started 61m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Container created 61m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 61m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Add eth0 [10.133.0.29/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 61m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Readiness probe failed: HTTP probe failed with statuscode: 503 61m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Readiness probe error: HTTP probe failed with statuscode: 503... 61m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-d695g 61m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-d695g Stopping container discovery 61m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 61m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 0 to 1 59m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-cngvb 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Add eth0 [10.133.0.40/23] from ovn-kubernetes 58m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Container created 58m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Container started 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Startup probe failed: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Startup probe error: Get "http://10.133.0.40:15021/healthz/ready": dial tcp 10.133.0.40:15021: connect: connection refused... 57m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-qcfkv 56m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 56m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-l4gst 56m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 56m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 56m Normal Pulling pod/payload-pre-processing-7b98bb678c-qcfkv Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-qcfkv Add eth0 [10.134.0.36/23] from ovn-kubernetes 56m Normal Pulling pod/payload-processing-5f78687d5b-l4gst Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal AddedInterface pod/payload-processing-5f78687d5b-l4gst Add eth0 [10.133.0.51/23] from ovn-kubernetes 56m Normal Pulling pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 56m Normal Created pod/payload-pre-processing-7b98bb678c-qcfkv Container created 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-lxc86 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-4nbkc 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Add eth0 [10.134.0.37/23] from ovn-kubernetes 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Add eth0 [10.132.0.40/23] from ovn-kubernetes 56m Normal Pulling pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 56m Normal Started pod/payload-processing-5f78687d5b-l4gst Container started 56m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 56m Normal Pulled pod/payload-processing-5f78687d5b-l4gst Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.004s (2.004s including waiting). Image size: 125317101 bytes. 56m Normal Started pod/payload-pre-processing-7b98bb678c-qcfkv Container started 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 1 to 3 56m Normal Created pod/payload-processing-5f78687d5b-l4gst Container created 56m Normal Pulled pod/payload-pre-processing-7b98bb678c-qcfkv Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.898s (1.898s including waiting). Image size: 125317101 bytes. 56m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Container created 56m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Container started 56m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.675s (2.675s including waiting). Image size: 183555333 bytes. 56m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Container started 56m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.911s (2.911s including waiting). Image size: 183555333 bytes. 56m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Container created 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Startup probe error: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused... 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Startup probe failed: Get "http://10.132.0.40:15021/healthz/ready": dial tcp 10.132.0.40:15021: connect: connection refused 56m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Container created 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-xx8n6 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-pdgh4 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Add eth0 [10.133.0.53/23] from ovn-kubernetes 56m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Container created 56m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Container started 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Add eth0 [10.134.0.38/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-ws84g 56m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Container created 56m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 3 to 6 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Add eth0 [10.132.0.41/23] from ovn-kubernetes 56m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Container started 56m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Container started 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Startup probe failed: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Startup probe error: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused... 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 56m Normal AddedLabel gateway/e2e-worker-w1-29be73 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-29be73 56m Normal AddedLabel gateway/e2e-worker-w0-52650f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-52650f 56m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b 56m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt 56m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled up replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 0 to 1 56m Normal AddedLabel gateway/e2e-worker-w3-cefb27 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-cefb27 56m Normal AddedLabel gateway/e2e-worker-w5-6f4fba Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-6f4fba 56m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled up replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 0 to 1 56m Normal AddedLabel gateway/e2e-worker-w4-ca5363 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-ca5363 56m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled up replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 0 to 1 56m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled up replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n 56m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled up replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 0 to 1 56m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled up replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 0 to 1 56m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 56m Normal AddedLabel gateway/e2e-worker-w2-1a11cf Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-1a11cf 56m Normal AddedLabel gateway/e2e-worker-w6-021d6d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-021d6d 56m Normal AddedInterface pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Add eth0 [10.134.0.39/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Created pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b 55m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Add eth0 [10.134.0.42/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Add eth0 [10.134.0.44/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Add eth0 [10.133.0.54/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Add eth0 [10.134.0.41/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x 55m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled up replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 0 to 1 55m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Add eth0 [10.134.0.43/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Add eth0 [10.134.0.40/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-cefb27 Scaled up replica set payload-processing-e2e-worker-w3-cefb27-88db76d58 from 0 to 1 55m Normal Pulled pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-cefb27-88db76d58 Created pod: payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75 Error creating: pods "payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-cefb27: serviceaccount "payload-processing-e2e-worker-w3-cefb27" not found 55m Normal Started pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Container started 55m Normal Created pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Container created 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-cefb27 Scaled up replica set payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75 from 0 to 1 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-cefb27-88db76d58 Error creating: pods "payload-processing-e2e-worker-w3-cefb27-88db76d58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-cefb27: serviceaccount "payload-processing-e2e-worker-w3-cefb27" not found 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Add eth0 [10.133.0.56/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75 Created pod: payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc 55m Normal AddedInterface pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Add eth0 [10.133.0.57/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-6f4fba Scaled up replica set payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7 from 0 to 1 55m Normal Created pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Container created 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-6f4fba Scaled up replica set payload-processing-e2e-worker-w5-6f4fba-86877b49f8 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7 Error creating: pods "payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-6f4fba: serviceaccount "payload-processing-e2e-worker-w5-6f4fba" not found 55m Normal Started pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Container started 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-6f4fba-86877b49f8 Error creating: pods "payload-processing-e2e-worker-w5-6f4fba-86877b49f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-6f4fba: serviceaccount "payload-processing-e2e-worker-w5-6f4fba" not found 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7 Created pod: payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh 55m Normal Started pod/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh Container started 55m Normal Created pod/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh Container created 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-ca5363-657fbd45cb Error creating: pods "payload-processing-e2e-worker-w4-ca5363-657fbd45cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-ca5363: serviceaccount "payload-processing-e2e-worker-w4-ca5363" not found 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-ca5363 Scaled up replica set payload-processing-e2e-worker-w4-ca5363-657fbd45cb from 0 to 1 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh Add eth0 [10.133.0.59/23] from ovn-kubernetes 55m Normal AddedInterface pod/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf Add eth0 [10.133.0.60/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf Container created 55m Normal Started pod/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf Container started 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-ca5363 Scaled up replica set payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-6f4fba-86877b49f8 Created pod: payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99 Created pod: payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99 Error creating: pods "payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-ca5363: serviceaccount "payload-processing-e2e-worker-w4-ca5363" not found 55m Normal Pulled pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-29be73 Scaled up replica set payload-pre-processing-e2e-worker-w1-29be73-64579fcddc from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-29be73-58c6d59956 Created pod: payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-29be73-58c6d59956 Error creating: pods "payload-processing-e2e-worker-w1-29be73-58c6d59956-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-29be73: serviceaccount "payload-processing-e2e-worker-w1-29be73" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc Created pod: payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l 55m Normal AddedInterface pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Add eth0 [10.133.0.62/23] from ovn-kubernetes 55m Normal Started pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Container started 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-ca5363-657fbd45cb Created pod: payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 55m Normal Created pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Container created 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Add eth0 [10.133.0.61/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc Error creating: pods "payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-29be73: serviceaccount "payload-processing-e2e-worker-w1-29be73" not found 55m Normal Created pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Container created 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-29be73 Scaled up replica set payload-processing-e2e-worker-w1-29be73-58c6d59956 from 0 to 1 55m Normal Started pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Container started 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Add eth0 [10.133.0.64/23] from ovn-kubernetes 55m Normal Started pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Container started 55m Normal Created pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Container created 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Add eth0 [10.133.0.63/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Container created 55m Normal Started pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Container started 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-52650f Scaled up replica set payload-pre-processing-e2e-worker-w0-52650f-7686c45df9 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9 Created pod: payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-52650f-5768bb8d4b Error creating: pods "payload-processing-e2e-worker-w0-52650f-5768bb8d4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-52650f: serviceaccount "payload-processing-e2e-worker-w0-52650f" not found 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9 Error creating: pods "payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-52650f: serviceaccount "payload-processing-e2e-worker-w0-52650f" not found 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-52650f Scaled up replica set payload-processing-e2e-worker-w0-52650f-5768bb8d4b from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-52650f-5768bb8d4b Created pod: payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-021d6d Scaled up replica set payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9 from 0 to 1 55m Normal Created pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Container created 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Add eth0 [10.134.0.45/23] from ovn-kubernetes 55m Normal Created pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Container created 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-021d6d Scaled up replica set payload-processing-e2e-worker-w6-021d6d-765d889d96 from 0 to 1 55m Normal AddedInterface pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Add eth0 [10.134.0.46/23] from ovn-kubernetes 55m Normal Started pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Container started 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Container started 55m Normal Pulled pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9 Error creating: pods "payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-021d6d: serviceaccount "payload-processing-e2e-worker-w6-021d6d" not found 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Add eth0 [10.134.0.47/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9 Error creating: pods "payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-1a11cf: serviceaccount "payload-processing-e2e-worker-w2-1a11cf" not found 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-1a11cf Scaled up replica set payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9 Created pod: payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 55m Normal Created pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Container created 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4 Error creating: pods "payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-1a11cf: serviceaccount "payload-processing-e2e-worker-w2-1a11cf" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-021d6d-765d889d96 Created pod: payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-1a11cf Scaled up replica set payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4 from 0 to 1 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-021d6d-765d889d96 Error creating: pods "payload-processing-e2e-worker-w6-021d6d-765d889d96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-021d6d: serviceaccount "payload-processing-e2e-worker-w6-021d6d" not found 55m Normal AddedInterface pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Add eth0 [10.134.0.48/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9 Created pod: payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b 55m Normal Started pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Container started 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4 Created pod: payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Add eth0 [10.133.0.67/23] from ovn-kubernetes 55m Normal Created pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Container created 55m Normal Created pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Container created 55m Normal Started pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Container started 55m Normal Started pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Container started 55m Normal AddedInterface pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Add eth0 [10.134.0.49/23] from ovn-kubernetes 55m Normal Started pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Container started 55m Normal Pulled pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Container created 55m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Readiness probe failed: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Readiness probe error: Get "http://10.134.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Normal Pulled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Container created 55m Normal Started pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Container started 55m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 55m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Container created 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Container started 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Container started 55m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Container started 55m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Container created 55m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Container created 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Readiness probe error: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 55m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Readiness probe failed: Get "http://10.134.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 55m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 55m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Container started 55m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Container created 55m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Container started 55m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Container created 55m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Container started 55m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Container created 55m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Container created 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled up replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 1 to 2 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Add eth0 [10.132.0.48/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Container created 54m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Container started 54m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Container started 54m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Add eth0 [10.132.0.47/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled up replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv 54m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Created pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled up replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 1 to 2 54m Normal Started pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Container started 54m Normal Pulled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Container created 54m Normal AddedInterface pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Add eth0 [10.132.0.51/23] from ovn-kubernetes 54m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Startup probe failed: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Startup probe error: Get "http://10.132.0.51:15021/healthz/ready": dial tcp 10.132.0.51:15021: connect: connection refused... 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled up replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 1 to 2 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled up replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Add eth0 [10.133.0.71/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Container started 54m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Container created 54m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Container created 54m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Add eth0 [10.132.0.52/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Container started 54m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Startup probe error: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Startup probe failed: Get "http://10.132.0.52:15021/healthz/ready": dial tcp 10.132.0.52:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled up replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 2 to 3 53m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Add eth0 [10.132.0.58/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled up replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 1 to 2 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Add eth0 [10.133.0.72/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Container created 53m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Container started 53m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Container created 53m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Container started 53m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled up replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb 53m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Container created 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled up replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 2 to 3 53m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Container created 53m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Add eth0 [10.133.0.74/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Add eth0 [10.134.0.74/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Add eth0 [10.133.0.73/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled up replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 2 to 4 53m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled up replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 3 to 4 53m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Container started 53m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Container started 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Add eth0 [10.133.0.75/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Add eth0 [10.134.0.75/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Container started 53m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Container created 53m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Container created 53m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Container started 53m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Container started 53m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Container created 53m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled up replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 1 to 2 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd 53m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Container started 53m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Container created 53m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Add eth0 [10.134.0.77/23] from ovn-kubernetes 53m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-s47sp 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-kfpcv 53m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-fwrpg 53m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Add eth0 [10.133.0.76/23] from ovn-kubernetes 53m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Container started 53m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Container started 53m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Container created 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Add eth0 [10.134.0.79/23] from ovn-kubernetes 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Add eth0 [10.132.0.62/23] from ovn-kubernetes 53m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Container created 53m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Container started 53m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Container started 53m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Add eth0 [10.134.0.78/23] from ovn-kubernetes 53m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Container created 53m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Add eth0 [10.132.0.63/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 52m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Container created 52m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Container started 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h 52m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Container started 52m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Add eth0 [10.134.0.80/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Container created 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Startup probe failed: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Startup probe error: Get "http://10.132.0.62:15021/healthz/ready": dial tcp 10.132.0.62:15021: connect: connection refused... 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 52m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled up replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 4 to 7 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled up replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 4 to 6 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 52m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Container created 52m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Add eth0 [10.132.0.65/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Add eth0 [10.133.0.78/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Container started 52m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Container created 52m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Container started 52m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Container created 52m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Container started 52m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Add eth0 [10.134.0.82/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Add eth0 [10.132.0.66/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Container started 52m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Container created 52m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Container started 52m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Add eth0 [10.133.0.77/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Container created 52m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Startup probe failed: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Startup probe error: Get "http://10.132.0.65:15021/healthz/ready": dial tcp 10.132.0.65:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled up replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 3 to 6 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 52m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Container created 52m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Container started 52m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Container started 52m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Add eth0 [10.133.0.80/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Add eth0 [10.134.0.84/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Container created 52m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Add eth0 [10.132.0.67/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Container created 52m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Container started 52m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Startup probe failed: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Startup probe error: Get "http://10.132.0.67:15021/healthz/ready": dial tcp 10.132.0.67:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled up replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 2 to 3 52m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Add eth0 [10.132.0.69/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Container created 52m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Container started 52m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled up replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 6 to 10 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled up replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 6 to 10 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled up replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 7 to 10 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 (combined from similar events): Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 (combined from similar events): Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Created pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Created pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 (combined from similar events): Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Created pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv 51m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Container started 51m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Container started 51m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Container started 51m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Add eth0 [10.132.0.71/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Container started 51m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Container started 51m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Add eth0 [10.133.0.83/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Container created 51m Normal Started pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Container started 51m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Container created 51m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Container created 51m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Add eth0 [10.134.0.93/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Container created 51m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Container started 51m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Container started 51m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Container created 51m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Add eth0 [10.133.0.84/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Container created 51m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Container created 51m Normal Started pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Container started 51m Normal Created pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Container created 51m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Add eth0 [10.134.0.92/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Container started 51m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Add eth0 [10.132.0.72/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Add eth0 [10.133.0.82/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Add eth0 [10.134.0.91/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Add eth0 [10.134.0.90/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Container created 51m Normal Started pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Container started 51m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Container created 51m Normal Pulled pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Container created 51m Normal Pulled pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Add eth0 [10.134.0.89/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Add eth0 [10.132.0.73/23] from ovn-kubernetes 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Readiness probe error: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Readiness probe failed: Get "http://10.132.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Readiness probe failed: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Readiness probe error: Get "http://10.132.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Readiness probe failed: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Readiness probe error: Get "http://10.133.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled up replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 3 to 6 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch 51m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Add eth0 [10.133.0.85/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Container started 51m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Container started 51m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Container created 51m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Container created 51m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 51m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Container created 51m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Add eth0 [10.132.0.75/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Container started 51m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Add eth0 [10.134.0.97/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Readiness probe failed: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled up replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 6 to 7 50m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Add eth0 [10.134.0.103/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Container created 50m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Container started 50m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Startup probe failed: Get "http://10.134.0.103:15021/healthz/ready": dial tcp 10.134.0.103:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Startup probe error: Get "http://10.134.0.103:15021/healthz/ready": dial tcp 10.134.0.103:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Readiness probe failed: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Container started 50m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Container created 50m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Container created 50m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled up replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 7 to 10 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 50m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Container started 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f (combined from similar events): Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Created pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt 50m Normal Started pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Container started 50m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Add eth0 [10.133.0.87/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Add eth0 [10.134.0.111/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Add eth0 [10.132.0.82/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Container created 50m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 49m Normal AddedLabel gateway/e2e-ait-3ba85458 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-3ba85458 49m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-j5x7c 49m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-ait-3ba85458-openshift-default Scaled up replica set e2e-ait-3ba85458-openshift-default-5f56f8659c from 0 to 1 49m Normal SuccessfulCreate replicaset/e2e-ait-3ba85458-openshift-default-5f56f8659c Created pod: e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq 49m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 49m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 49m Normal AddedInterface pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Add eth0 [10.134.0.113/23] from ovn-kubernetes 49m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Add eth0 [10.132.0.84/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7 Created pod: e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s 49m Normal AddedLabel gateway/e2e-migrate-cab9990d-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-cab9990d-gw 49m Normal AddedInterface pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Add eth0 [10.134.0.115/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-migrate-cab9990d-gw-openshift-default Scaled up replica set e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-3ba85458-646f477df9 Created pod: payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-3ba85458 Scaled up replica set payload-processing-e2e-ait-3ba85458-7cf86cd866 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-3ba85458-7cf86cd866 Created pod: payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 49m Warning FailedCreate replicaset/payload-processing-e2e-ait-3ba85458-7cf86cd866 Error creating: pods "payload-processing-e2e-ait-3ba85458-7cf86cd866-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-3ba85458: serviceaccount "payload-processing-e2e-ait-3ba85458" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-3ba85458 Scaled up replica set payload-pre-processing-e2e-ait-3ba85458-646f477df9 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-3ba85458-646f477df9 Error creating: pods "payload-pre-processing-e2e-ait-3ba85458-646f477df9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-3ba85458: serviceaccount "payload-processing-e2e-ait-3ba85458" not found 49m Normal Created pod/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 Container created 49m Normal Pulled pod/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Pulled pod/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc Container started 49m Normal AddedInterface pod/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc Add eth0 [10.133.0.91/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 Container started 49m Normal Created pod/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc Container created 49m Normal AddedInterface pod/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 Add eth0 [10.133.0.92/23] from ovn-kubernetes 49m Warning FailedCreate replicaset/payload-processing-e2e-migrate-cab9990d-75bddb75b6 Error creating: pods "payload-processing-e2e-migrate-cab9990d-75bddb75b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-cab9990d: serviceaccount "payload-processing-e2e-migrate-cab9990d" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-cab9990d Scaled up replica set payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-cab9990d-75bddb75b6 Created pod: payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf Error creating: pods "payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-cab9990d: serviceaccount "payload-processing-e2e-migrate-cab9990d" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf Created pod: payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-cab9990d Scaled up replica set payload-processing-e2e-migrate-cab9990d-75bddb75b6 from 0 to 1 49m Normal Pulled pod/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn Container created 49m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Container started 49m Normal Started pod/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn Container started 49m Normal Created pod/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn Container created 49m Normal AddedInterface pod/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn Add eth0 [10.133.0.94/23] from ovn-kubernetes 49m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn Add eth0 [10.133.0.93/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn Container started 49m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 7.793s (7.793s including waiting). Image size: 185476477 bytes. 49m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Container created 49m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Readiness probe failed: HTTP probe failed with statuscode: 503 49m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-j5x7c Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-3ba85458-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-3ba85458-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled up replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 2 to 3 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Created pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 49m Normal Started pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Container started 49m Normal Created pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Container created 49m Normal AddedInterface pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Add eth0 [10.133.0.96/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-3ba85458-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-3ba85458-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-cab9990d-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-cab9990d-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled up replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 4 to 8 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx 49m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb 49m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Add eth0 [10.134.0.120/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Add eth0 [10.132.0.85/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Add eth0 [10.132.0.86/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Container created 49m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Container started 49m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Container created 49m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Add eth0 [10.133.0.98/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Container created 49m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Container started 49m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Container created 49m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Container started 49m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Container started 49m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 49m Normal SuccessfulCreate replicaset/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7 Created pod: e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 49m Normal SuccessfulCreate replicaset/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7 Created pod: e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf 49m Normal SuccessfulCreate replicaset/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7 Created pod: e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-migrate-cab9990d-gw-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-migrate-cab9990d-gw-openshift-default Scaled up replica set e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7 from 1 to 4 49m Warning ProbeError pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Readiness probe error: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Readiness probe failed: Get "http://10.134.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Normal Pulled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 49m Normal Created pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Container created 49m Normal Pulled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Add eth0 [10.134.0.122/23] from ovn-kubernetes 49m Normal Started pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Container started 49m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Startup probe error: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused... 49m Normal Pulled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 49m Normal AddedInterface pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Add eth0 [10.133.0.99/23] from ovn-kubernetes 49m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Startup probe failed: Get "http://10.132.0.86:15021/healthz/ready": dial tcp 10.132.0.86:15021: connect: connection refused 49m Normal Created pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Container created 49m Normal Started pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Container started 49m Normal AddedInterface pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Add eth0 [10.132.0.88/23] from ovn-kubernetes 49m Normal Started pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Container started 49m Normal Created pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Container created 49m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Readiness probe error: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Readiness probe failed: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled up replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 8 to 9 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 9; reason: All metrics below target 48m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Container created 48m Warning Unhealthy pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 48m Warning ProbeError pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Startup probe error: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Startup probe failed: Get "http://10.133.0.99:15021/healthz/ready": dial tcp 10.133.0.99:15021: connect: connection refused 48m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Add eth0 [10.133.0.100/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 48m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Container started 48m Normal Killing pod/payload-pre-processing-e2e-ait-3ba85458-646f477df9-4xctc Stopping container payload-pre-processing 48m Normal Started pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Container started 48m Normal Created pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Container created 48m Normal Killing pod/payload-processing-e2e-ait-3ba85458-7cf86cd866-6xz69 Stopping container payload-processing 48m Normal Pulled pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Container started 48m Normal Created pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Container created 48m Normal Pulled pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Killing pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Stopping container istio-proxy 48m Normal AddedLabel gateway/e2e-ait-751d6789 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-751d6789 48m Warning Unhealthy pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 48m Normal ScalingReplicaSet deployment/e2e-ait-751d6789-openshift-default Scaled up replica set e2e-ait-751d6789-openshift-default-986c6d7 from 0 to 1 48m Normal Killing pod/payload-pre-processing-e2e-migrate-cab9990d-76b4b665cf-75mnn Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-migrate-cab9990d-75bddb75b6-nsjnn Stopping container payload-processing 48m Warning ProbeError pod/e2e-ait-3ba85458-openshift-default-5f56f8659c-fsdfq Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 48m Normal SuccessfulCreate replicaset/e2e-ait-751d6789-openshift-default-986c6d7 Created pod: e2e-ait-751d6789-openshift-default-986c6d7-lfbwt 48m Normal AddedInterface pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Add eth0 [10.134.0.123/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 48m Warning ProbeError pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Startup probe failed: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Startup probe error: Get "http://10.133.0.100:15021/healthz/ready": dial tcp 10.133.0.100:15021: connect: connection refused... 48m Normal Killing pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-b8cmf Stopping container istio-proxy 48m Normal Killing pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-c85z8 Stopping container istio-proxy 48m Normal Killing pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-vb944 Stopping container istio-proxy 48m Normal Killing pod/e2e-migrate-cab9990d-gw-openshift-default-77d8f976f7-mfs2s Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-751d6789 Scaled up replica set payload-pre-processing-e2e-ait-751d6789-7fb4c76b79 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79 Error creating: pods "payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-751d6789: serviceaccount "payload-processing-e2e-ait-751d6789" not found 48m Normal SuccessfulCreate replicaset/e2e-derive-be862c05-openshift-default-546c6fcf65 Created pod: e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-751d6789 Scaled up replica set payload-processing-e2e-ait-751d6789-7bd9589f56 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-ait-751d6789-7bd9589f56 Error creating: pods "payload-processing-e2e-ait-751d6789-7bd9589f56-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-751d6789: serviceaccount "payload-processing-e2e-ait-751d6789" not found 48m Normal AddedLabel gateway/e2e-derive-be862c05 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-be862c05 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-751d6789-7bd9589f56 Created pod: payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79 Created pod: payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 48m Normal ScalingReplicaSet deployment/e2e-derive-be862c05-openshift-default Scaled up replica set e2e-derive-be862c05-openshift-default-546c6fcf65 from 0 to 1 48m Normal AddedInterface pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Add eth0 [10.134.0.125/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 Container started 48m Normal Pulled pod/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 Container created 48m Normal Started pod/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 Container started 48m Normal AddedInterface pod/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 Add eth0 [10.133.0.102/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 Add eth0 [10.133.0.101/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 Container created 48m Normal Pulled pod/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Created pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Created pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 48m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled up replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 3 to 6 48m Normal SuccessfulCreate replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Created pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 48m Normal Started pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Container started 48m Normal Pulled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-be862c05-c76879479 Error creating: pods "payload-pre-processing-e2e-derive-be862c05-c76879479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-be862c05: serviceaccount "payload-processing-e2e-derive-be862c05" not found 48m Normal AddedInterface pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Add eth0 [10.132.0.89/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Container created 48m Normal AddedInterface pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Add eth0 [10.133.0.103/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Add eth0 [10.134.0.126/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-be862c05-c76879479 Created pod: payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-be862c05 Scaled up replica set payload-pre-processing-e2e-derive-be862c05-c76879479 from 0 to 1 48m Normal Created pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Container created 48m Warning FailedCreate replicaset/payload-processing-e2e-derive-be862c05-6b54db7858 Error creating: pods "payload-processing-e2e-derive-be862c05-6b54db7858-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-be862c05: serviceaccount "payload-processing-e2e-derive-be862c05" not found 48m Normal Started pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Container started 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-be862c05-6b54db7858 Created pod: payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-be862c05 Scaled up replica set payload-processing-e2e-derive-be862c05-6b54db7858 from 0 to 1 48m Normal Created pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Container created 48m Normal Pulled pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Started pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Container started 48m Normal Pulled pod/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r Add eth0 [10.133.0.105/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h Add eth0 [10.133.0.104/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r Container created 48m Normal Started pod/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h Container started 48m Normal Started pod/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r Container started 48m Normal Created pod/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h Container created 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-751d6789-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-751d6789-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Startup probe error: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Startup probe failed: Get "http://10.134.0.126:15021/healthz/ready": dial tcp 10.134.0.126:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled up replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 9 to 10 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf (combined from similar events): Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw 48m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc 48m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-thfqc Stopping container istio-proxy 48m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Container created 48m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Add eth0 [10.134.0.127/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Container started 48m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-751d6789-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-751d6789-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-be862c05-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-be862c05-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedLabel gateway/e2e-shared-a-w3-11f521 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-11f521 48m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-11f521-openshift-default-96689b67c Created pod: e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-11f521-openshift-default Scaled up replica set e2e-shared-a-w3-11f521-openshift-default-96689b67c from 0 to 1 48m Normal AddedInterface pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Add eth0 [10.134.0.128/23] from ovn-kubernetes 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78 Error creating: pods "payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-11f521: serviceaccount "payload-processing-e2e-shared-a-w3-11f521" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75 Created pod: payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-11f521 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-11f521 Scaled up replica set payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78 Created pod: payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-11f521: serviceaccount "payload-processing-e2e-shared-a-w3-11f521" not found 48m Normal Created pod/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq Container created 48m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq Container started 48m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 Container created 48m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq Add eth0 [10.133.0.109/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 Add eth0 [10.133.0.108/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Readiness probe failed: Get "http://10.134.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Readiness probe error: Get "http://10.134.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal ScalingReplicaSet deployment/e2e-derive-be862c05-openshift-default Scaled up replica set e2e-derive-be862c05-openshift-default-546c6fcf65 from 1 to 2 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-derive-be862c05-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal Killing pod/payload-pre-processing-e2e-ait-751d6789-7fb4c76b79-l6pp8 Stopping container payload-pre-processing 48m Normal AddedInterface pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Add eth0 [10.132.0.91/23] from ovn-kubernetes 48m Normal Killing pod/payload-processing-e2e-ait-751d6789-7bd9589f56-8vx54 Stopping container payload-processing 48m Normal SuccessfulCreate replicaset/e2e-derive-be862c05-openshift-default-546c6fcf65 Created pod: e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 48m Normal Pulled pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Container created 48m Normal Started pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Container started 48m Normal Created pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Container created 48m Normal Started pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Container started 48m Normal Pulled pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Readiness probe failed: Get "http://10.134.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Readiness probe error: Get "http://10.134.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled up replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 2 to 4 47m Normal Created pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Container created 47m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Add eth0 [10.132.0.92/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Add eth0 [10.134.0.131/23] from ovn-kubernetes 47m Normal Started pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Container started 47m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Container created 47m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Container started 47m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Container created 47m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Container started 47m Warning Unhealthy pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 47m Warning ProbeError pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 47m Normal Killing pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 47m Warning ProbeError pod/e2e-ait-751d6789-openshift-default-986c6d7-lfbwt Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5f387c-openshift-default Scaled up replica set e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59 from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59 Created pod: e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n 47m Normal AddedLabel gateway/e2e-shared-a-w0-5f387c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-5f387c 47m Normal AddedInterface pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Add eth0 [10.134.0.133/23] from ovn-kubernetes 47m Normal Killing pod/payload-processing-e2e-derive-be862c05-6b54db7858-9bx9r Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-derive-be862c05-c76879479-bfz2h Stopping container payload-pre-processing 47m Warning ProbeError pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused 47m Normal Killing pod/e2e-derive-be862c05-openshift-default-546c6fcf65-zvh4m Stopping container istio-proxy 47m Normal Killing pod/e2e-derive-be862c05-openshift-default-546c6fcf65-kfvq9 Stopping container istio-proxy 47m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-gfvgn 47m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 47m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 10 to 6 47m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-ws84g Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-ws84g 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-s47sp 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-lxc86 47m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-s47sp Stopping container istio-proxy 47m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-lxc86 Stopping container istio-proxy 47m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-gfvgn Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96 Created pod: payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-5f387c Scaled up replica set payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99 Created pod: payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-5f387c: serviceaccount "payload-processing-e2e-shared-a-w0-5f387c" not found 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96 Error creating: pods "payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-5f387c: serviceaccount "payload-processing-e2e-shared-a-w0-5f387c" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-5f387c Scaled up replica set payload-processing-e2e-shared-a-w0-5f387c-d78d85d96 from 0 to 1 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl Container started 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl Add eth0 [10.133.0.111/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl Container created 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl Add eth0 [10.133.0.112/23] from ovn-kubernetes 47m Normal Created pod/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl Container created 47m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled up replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 4 to 5 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 5; reason: 47m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z 47m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Add eth0 [10.132.0.93/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Container created 47m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Container started 47m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-mt-d7a6db25-openshift-default-68896bcdc7 Created pod: e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg 47m Normal ScalingReplicaSet deployment/e2e-mt-d7a6db25-openshift-default Scaled up replica set e2e-mt-d7a6db25-openshift-default-68896bcdc7 from 0 to 1 47m Normal AddedLabel gateway/e2e-mt-d7a6db25 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-d7a6db25 47m Normal AddedInterface pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Add eth0 [10.134.0.135/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d7a6db25 Scaled up replica set payload-processing-e2e-mt-d7a6db25-56b6fb9474 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d7a6db25 Scaled up replica set payload-pre-processing-e2e-mt-d7a6db25-67ddd99559 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-mt-d7a6db25-56b6fb9474 Error creating: pods "payload-processing-e2e-mt-d7a6db25-56b6fb9474-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d7a6db25: serviceaccount "payload-processing-e2e-mt-d7a6db25" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d7a6db25-56b6fb9474 Created pod: payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn 47m Normal Started pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Add eth0 [10.133.0.114/23] from ovn-kubernetes 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559 Error creating: pods "payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d7a6db25: serviceaccount "payload-processing-e2e-mt-d7a6db25" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559 Created pod: payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf 47m Normal Created pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Container created 47m Normal Pulled pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Pulled pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Container started 47m Normal Created pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Container created 47m Normal AddedInterface pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Add eth0 [10.134.0.136/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Container created 47m Normal Started pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Container started 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7 Created pod: e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q 47m Normal AddedLabel gateway/e2e-shared-b-w3-8e0f66 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-8e0f66 47m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-8e0f66-openshift-default Scaled up replica set e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7 from 0 to 1 47m Normal AddedInterface pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Add eth0 [10.134.0.137/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Startup probe error: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Startup probe failed: Get "http://10.134.0.128:15021/healthz/ready": dial tcp 10.134.0.128:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-8e0f66 Scaled up replica set payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd Error creating: pods "payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8e0f66: serviceaccount "payload-processing-e2e-shared-b-w3-8e0f66" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc Created pod: payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc Error creating: pods "payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8e0f66: serviceaccount "payload-processing-e2e-shared-b-w3-8e0f66" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-8e0f66 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc from 0 to 1 47m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Add eth0 [10.133.0.116/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd Created pod: payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Add eth0 [10.133.0.115/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Container created 47m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Container started 47m Normal Pulled pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Container created 47m Normal Started pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Container started 47m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf Created pod: e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms 47m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-00e07a-openshift-default Scaled up replica set e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf from 0 to 1 47m Normal AddedLabel gateway/e2e-shared-b-w0-00e07a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-00e07a 47m Normal AddedInterface pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Add eth0 [10.134.0.138/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Readiness probe error: Get "http://10.134.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Readiness probe failed: Get "http://10.134.0.133:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Pulled pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-00e07a Scaled up replica set payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48 from 0 to 1 47m Normal Started pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Container started 47m Normal Created pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Container created 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-00e07a Scaled up replica set payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7 Created pod: payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48 Created pod: payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48 Error creating: pods "payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-00e07a: serviceaccount "payload-processing-e2e-shared-b-w0-00e07a" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-00e07a: serviceaccount "payload-processing-e2e-shared-b-w0-00e07a" not found 47m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr Container created 47m Normal Pulled pod/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr Container started 47m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v Add eth0 [10.134.0.140/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v Container started 47m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal Created pod/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v Container created 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr Add eth0 [10.134.0.139/23] from ovn-kubernetes 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-8e0f66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-8e0f66-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning Unhealthy pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Startup probe failed: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Startup probe error: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-5f387c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-5f387c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c (combined from similar events): Created pod: maas-default-gateway-openshift-default-7d9897b54c-jwx9q 46m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Container created 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Add eth0 [10.132.0.94/23] from ovn-kubernetes 46m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Container started 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Add eth0 [10.134.0.142/23] from ovn-kubernetes 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Add eth0 [10.134.0.141/23] from ovn-kubernetes 46m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Container created 46m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Container started 46m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Container created 46m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Container started 46m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Container created 46m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Container started 46m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Add eth0 [10.134.0.143/23] from ovn-kubernetes 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Readiness probe error: Get "http://10.134.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal SuccessfulCreate replicaset/e2e-mt-d7a6db25-openshift-default-68896bcdc7 Created pod: e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv 46m Warning Unhealthy pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Readiness probe failed: Get "http://10.134.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal ScalingReplicaSet deployment/e2e-mt-d7a6db25-openshift-default Scaled up replica set e2e-mt-d7a6db25-openshift-default-68896bcdc7 from 1 to 3 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-d7a6db25-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 46m Normal SuccessfulCreate replicaset/e2e-mt-d7a6db25-openshift-default-68896bcdc7 Created pod: e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg 46m Normal Created pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Container created 46m Normal Pulled pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Add eth0 [10.133.0.120/23] from ovn-kubernetes 46m Normal Started pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Container started 46m Normal Created pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Container created 46m Normal Started pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Container started 46m Normal AddedInterface pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Add eth0 [10.132.0.96/23] from ovn-kubernetes 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 46m Normal ScalingReplicaSet deployment/e2e-api-a-1fa555-openshift-default Scaled up replica set e2e-api-a-1fa555-openshift-default-574587c9ff from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-api-a-1fa555-openshift-default-574587c9ff Created pod: e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 46m Normal AddedLabel gateway/e2e-api-a-1fa555 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-1fa555 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Startup probe failed: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Startup probe error: Get "http://10.134.0.142:15021/healthz/ready": dial tcp 10.134.0.142:15021: connect: connection refused... 46m Normal AddedInterface pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Add eth0 [10.134.0.144/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Readiness probe error: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Readiness probe failed: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 46m Warning ProbeError pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 46m Normal Pulled pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Container created 46m Normal Pulled pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Container started 46m Normal Started pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Container started 46m Normal Created pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Container created 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-1fa555 Scaled up replica set payload-processing-e2e-api-a-1fa555-7576b49db4 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-1fa555 Scaled up replica set payload-pre-processing-e2e-api-a-1fa555-54499dd979 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-1fa555-7576b49db4 Created pod: payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 46m Warning FailedCreate replicaset/payload-processing-e2e-api-a-1fa555-7576b49db4 Error creating: pods "payload-processing-e2e-api-a-1fa555-7576b49db4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1fa555: serviceaccount "payload-processing-e2e-api-a-1fa555" not found 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-1fa555-54499dd979 Error creating: pods "payload-pre-processing-e2e-api-a-1fa555-54499dd979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-1fa555: serviceaccount "payload-processing-e2e-api-a-1fa555" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-1fa555-54499dd979 Created pod: payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 46m Normal Started pod/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 Container started 46m Normal Pulled pod/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 Add eth0 [10.133.0.121/23] from ovn-kubernetes 46m Normal AddedInterface pod/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 Add eth0 [10.133.0.122/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 Container created 46m Normal Created pod/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 Container created 46m Normal Started pod/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 Container started 46m Warning ProbeError pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d7a6db25-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d7a6db25-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-5f387c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-5f387c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-d7a6db25-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-d7a6db25-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Pulled pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Container started 46m Normal Created pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Container created 46m Warning Unhealthy pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 46m Normal SuccessfulCreate replicaset/e2e-api-a-d788ba-openshift-default-7f77dc57fc Created pod: e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s 46m Normal ScalingReplicaSet deployment/e2e-api-a-d788ba-openshift-default Scaled up replica set e2e-api-a-d788ba-openshift-default-7f77dc57fc from 0 to 1 46m Normal AddedLabel gateway/e2e-api-a-d788ba Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d788ba 46m Normal AddedInterface pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Add eth0 [10.134.0.150/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-8e0f66-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-8e0f66-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8 Error creating: pods "payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d788ba: serviceaccount "payload-processing-e2e-api-a-d788ba" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d788ba Scaled up replica set payload-processing-e2e-api-a-d788ba-5fcb7b4fc8 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d788ba Scaled up replica set payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95 Created pod: payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95 Error creating: pods "payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d788ba: serviceaccount "payload-processing-e2e-api-a-d788ba" not found 45m Normal Started pod/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x Container started 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8 Created pod: payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x Add eth0 [10.134.0.151/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x Container created 45m Normal AddedInterface pod/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw Add eth0 [10.134.0.152/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw Container created 45m Normal Started pod/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw Container started 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-00e07a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-00e07a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal AddedLabel gateway/e2e-api-b-bdc256 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-bdc256 45m Normal ScalingReplicaSet deployment/e2e-api-b-bdc256-openshift-default Scaled up replica set e2e-api-b-bdc256-openshift-default-5cf7d97495 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-api-b-bdc256-openshift-default-5cf7d97495 Created pod: e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr 45m Normal AddedInterface pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Add eth0 [10.132.0.98/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-00e07a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-00e07a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Pulled pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-bdc256-66b6b9d4f6 Created pod: payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 45m Warning ProbeError pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Readiness probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 45m Warning FailedCreate replicaset/payload-processing-e2e-api-b-bdc256-66b6b9d4f6 Error creating: pods "payload-processing-e2e-api-b-bdc256-66b6b9d4f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bdc256: serviceaccount "payload-processing-e2e-api-b-bdc256" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-bdc256 Scaled up replica set payload-processing-e2e-api-b-bdc256-66b6b9d4f6 from 0 to 1 45m Warning Unhealthy pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Readiness probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-bdc256 Scaled up replica set payload-pre-processing-e2e-api-b-bdc256-7559f49547 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-bdc256-7559f49547 Created pod: payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-bdc256-7559f49547 Error creating: pods "payload-pre-processing-e2e-api-b-bdc256-7559f49547-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bdc256: serviceaccount "payload-processing-e2e-api-b-bdc256" not found 45m Normal Started pod/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv Container started 45m Normal Created pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Container created 45m Normal Pulled pod/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv Container created 45m Normal AddedInterface pod/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 Add eth0 [10.134.0.155/23] from ovn-kubernetes 45m Normal Started pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv Add eth0 [10.134.0.156/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 Container started 45m Normal Created pod/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 Container created 45m Normal Pulled pod/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d788ba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d788ba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 45m Normal Killing pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Stopping container payload-processing 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-1fa555-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-1fa555-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedLabel gateway/e2e-mt-52785f5e Added label istio.io/rev=openshift-gateway to gateway e2e-mt-52785f5e 45m Normal ScalingReplicaSet deployment/e2e-mt-52785f5e-openshift-default Scaled up replica set e2e-mt-52785f5e-openshift-default-69c968b885 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-mt-52785f5e-openshift-default-69c968b885 Created pod: e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm 45m Warning Unhealthy pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-khvkn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 45m Warning Unhealthy pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-q78mf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Readiness probe failed: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Readiness probe failed: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Readiness probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Readiness probe failed: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Readiness probe error: Get "http://10.132.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal AddedInterface pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Add eth0 [10.134.0.160/23] from ovn-kubernetes 45m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Readiness probe error: Get "http://10.132.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-52785f5e Scaled up replica set payload-pre-processing-e2e-mt-52785f5e-67559cc487 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-mt-52785f5e-5f57cff566 Error creating: pods "payload-processing-e2e-mt-52785f5e-5f57cff566-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-52785f5e: serviceaccount "payload-processing-e2e-mt-52785f5e" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-52785f5e Scaled up replica set payload-processing-e2e-mt-52785f5e-5f57cff566 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-52785f5e-5f57cff566 Created pod: payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-52785f5e-67559cc487 Created pod: payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-52785f5e-67559cc487 Error creating: pods "payload-pre-processing-e2e-mt-52785f5e-67559cc487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-52785f5e: serviceaccount "payload-processing-e2e-mt-52785f5e" not found 45m Normal AddedInterface pod/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 Add eth0 [10.134.0.161/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d Add eth0 [10.134.0.162/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d Container created 45m Normal Pulled pod/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 Container started 45m Normal Created pod/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 Container created 45m Normal Started pod/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d Container started 45m Normal Pulled pod/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedLabel gateway/e2e-api-b-02fbf5 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-02fbf5 44m Normal Pulled pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Container created 44m Normal SuccessfulCreate replicaset/e2e-api-b-02fbf5-openshift-default-64fc59d485 Created pod: e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc 44m Normal ScalingReplicaSet deployment/e2e-api-b-02fbf5-openshift-default Scaled up replica set e2e-api-b-02fbf5-openshift-default-64fc59d485 from 0 to 1 44m Normal Started pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Container started 44m Normal AddedInterface pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Add eth0 [10.132.0.99/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95 Error creating: pods "payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-02fbf5: serviceaccount "payload-processing-e2e-api-b-02fbf5" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-02fbf5 Scaled up replica set payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-api-b-02fbf5-6b759bfc8 Error creating: pods "payload-processing-e2e-api-b-02fbf5-6b759bfc8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-02fbf5: serviceaccount "payload-processing-e2e-api-b-02fbf5" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-02fbf5-6b759bfc8 Created pod: payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-02fbf5 Scaled up replica set payload-processing-e2e-api-b-02fbf5-6b759bfc8 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95 Created pod: payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 44m Warning Unhealthy pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Startup probe failed: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused 44m Normal Pulling pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal AddedInterface pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Add eth0 [10.132.0.101/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Startup probe error: Get "http://10.134.0.150:15021/healthz/ready": dial tcp 10.134.0.150:15021: connect: connection refused... 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-1fa555-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Pulling pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Add eth0 [10.132.0.100/23] from ovn-kubernetes 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-1fa555-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Pulled pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 6.124s (6.124s including waiting). Image size: 125317101 bytes. 44m Normal Started pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Container started 44m Normal Pulled pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 5.809s (5.809s including waiting). Image size: 125317101 bytes. 44m Normal Created pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Container created 44m Normal Started pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Container started 44m Normal Created pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Container created 44m Normal Created pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Container created 44m Normal Started pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Container started 44m Normal Pulled pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-d7a6db25-56b6fb9474 Created pod: payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-d7a6db25-56b6fb9474 Error creating: pods "payload-processing-e2e-mt-d7a6db25-56b6fb9474-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d7a6db25: serviceaccount "payload-processing-e2e-mt-d7a6db25" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-d7a6db25 Scaled up replica set payload-pre-processing-e2e-mt-d7a6db25-67ddd99559 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559 Error creating: pods "payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-d7a6db25: serviceaccount "payload-processing-e2e-mt-d7a6db25" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-d7a6db25 Scaled up replica set payload-processing-e2e-mt-d7a6db25-56b6fb9474 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559 Created pod: payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 44m Normal AddedInterface pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 Add eth0 [10.133.0.127/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 Add eth0 [10.134.0.163/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 Container created 44m Normal Started pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 Container started 44m Normal Created pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 Container created 44m Normal Pulled pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning ProbeError pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d788ba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d788ba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled up replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 5 to 8 44m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn 44m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 44m Normal SuccessfulCreate replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Created pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 44m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Add eth0 [10.133.0.128/23] from ovn-kubernetes 44m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd 44m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j Stopping container istio-proxy 44m Normal Killing pod/payload-processing-e2e-api-a-1fa555-7576b49db4-bh5h6 Stopping container payload-processing 44m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-24zch Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 5; reason: All metrics below target 44m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Container started 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled down replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 10 to 5 44m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Container started 44m Normal Killing pod/payload-pre-processing-e2e-api-a-1fa555-54499dd979-2gck7 Stopping container payload-pre-processing 44m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n Stopping container istio-proxy 44m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Add eth0 [10.134.0.164/23] from ovn-kubernetes 44m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Container created 44m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Container created 44m Normal Created pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Container created 44m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-586bd Stopping container istio-proxy 44m Normal Started pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Container started 44m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-4pk2n 44m Normal Pulled pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-m95bd 44m Normal AddedInterface pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Add eth0 [10.132.0.103/23] from ovn-kubernetes 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-gjq9j 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bdc256-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bdc256-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal AddedLabel gateway/e2e-inf-a-6444d7 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-6444d7 44m Normal ScalingReplicaSet deployment/e2e-inf-a-6444d7-openshift-default Scaled up replica set e2e-inf-a-6444d7-openshift-default-7bf76bdcd9 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9 Created pod: e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 44m Normal AddedInterface pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Add eth0 [10.132.0.104/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-6444d7 Scaled up replica set payload-processing-e2e-inf-a-6444d7-767fdb56d6 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999 Error creating: pods "payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6444d7: serviceaccount "payload-processing-e2e-inf-a-6444d7" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-6444d7 Scaled up replica set payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999 Created pod: payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-6444d7-767fdb56d6 Error creating: pods "payload-processing-e2e-inf-a-6444d7-767fdb56d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-6444d7: serviceaccount "payload-processing-e2e-inf-a-6444d7" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-6444d7-767fdb56d6 Created pod: payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh 44m Normal Started pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Container started 44m Normal Pulled pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Add eth0 [10.134.0.165/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Container created 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Container started 44m Normal Created pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Container created 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Add eth0 [10.134.0.166/23] from ovn-kubernetes 44m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r 44m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5mg6r Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled down replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 10 to 6 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 6; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-sb97m 44m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-r98qq Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bdc256-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bdc256-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-5c75b Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal Started pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Container started 43m Normal Pulled pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Container created 43m Normal Created pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Container created 43m Normal Pulled pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Container started 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6444d7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6444d7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 43m Warning ProbeError pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 43m Normal AddedLabel gateway/e2e-api-a-fa8dd0 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-fa8dd0 43m Warning ProbeError pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused 43m Normal ScalingReplicaSet deployment/e2e-api-a-fa8dd0-openshift-default Scaled up replica set e2e-api-a-fa8dd0-openshift-default-6588f97d75 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-api-a-fa8dd0-openshift-default-6588f97d75 Created pod: e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc 43m Normal AddedInterface pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Add eth0 [10.134.0.168/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c Error creating: pods "payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-fa8dd0: serviceaccount "payload-processing-e2e-api-a-fa8dd0" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-fa8dd0 Scaled up replica set payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd Error creating: pods "payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-fa8dd0: serviceaccount "payload-processing-e2e-api-a-fa8dd0" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd Created pod: payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-fa8dd0 Scaled up replica set payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c Created pod: payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b 43m Normal Started pod/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp Container started 43m Normal Created pod/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b Add eth0 [10.134.0.170/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp Container created 43m Normal Started pod/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b Container started 43m Normal AddedInterface pod/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp Add eth0 [10.134.0.171/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-02fbf5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-02fbf5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 43m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-nbcn8 Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-prznh 43m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb Stopping container istio-proxy 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 6; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-k46bb 43m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled down replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 10 to 6 43m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-vqnx5 Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-cgv7x Stopping container istio-proxy 43m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lmgj5 Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled down replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 10 to 7 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 7; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf 43m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-825hf Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-52785f5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/e2e-api-a-1fa555-openshift-default-574587c9ff-h28g4 Stopping container istio-proxy 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-52785f5e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Killing pod/payload-pre-processing-e2e-api-a-d788ba-5ff46bcf95-hlr4x Stopping container payload-pre-processing 43m Normal Killing pod/payload-processing-e2e-api-a-d788ba-5fcb7b4fc8-n8fvw Stopping container payload-processing 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-02fbf5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-02fbf5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Killing pod/payload-pre-processing-e2e-api-b-bdc256-7559f49547-94jpv Stopping container payload-pre-processing 43m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled down replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 6 to 4 43m Normal Killing pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 Stopping container istio-proxy 43m Normal Killing pod/payload-processing-e2e-api-b-bdc256-66b6b9d4f6-6sjc2 Stopping container payload-processing 43m Normal Killing pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default New size: 4; reason: All metrics below target 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Deleted pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-rttg5 43m Normal SuccessfulDelete replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Deleted pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-t6pxx 43m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd 43m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-tlfnd Stopping container istio-proxy 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 6; reason: All metrics below target 43m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled down replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 7 to 6 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-52785f5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-52785f5e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 43m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-5gbdq 43m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 43m Warning ProbeError pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Readiness probe error: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Readiness probe failed: Get "http://10.132.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Add eth0 [10.134.0.174/23] from ovn-kubernetes 43m Normal Killing pod/payload-processing-e2e-mt-d7a6db25-56b6fb9474-6fpr2 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-mt-d7a6db25-67ddd99559-bpxv4 Stopping container payload-pre-processing 43m Normal Pulled pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.759s (2.759s including waiting). Image size: 185476477 bytes. 43m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Container started 43m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Container created 43m Normal Created pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Container created 43m Normal Started pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Container started 43m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal AddedInterface pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Add eth0 [10.134.0.176/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Startup probe failed: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Startup probe error: Get "http://10.132.0.104:15021/healthz/ready": dial tcp 10.132.0.104:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/e2e-inf-b-b3bdf9-openshift-default-755bccddf Created pod: e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 43m Normal ScalingReplicaSet deployment/e2e-inf-b-b3bdf9-openshift-default Scaled up replica set e2e-inf-b-b3bdf9-openshift-default-755bccddf from 0 to 1 43m Normal AddedLabel gateway/e2e-inf-b-b3bdf9 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-b3bdf9 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c Error creating: pods "payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-b3bdf9: serviceaccount "payload-processing-e2e-inf-b-b3bdf9" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8 Created pod: payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-b3bdf9 Scaled up replica set payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c Created pod: payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-b3bdf9 Scaled up replica set payload-pre-processing-e2e-inf-b-b3bdf9-867787698c from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8 Error creating: pods "payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-b3bdf9: serviceaccount "payload-processing-e2e-inf-b-b3bdf9" not found 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Container started 43m Normal Started pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Add eth0 [10.134.0.178/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Add eth0 [10.134.0.177/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Container created 43m Normal Created pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Container created 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-6444d7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-6444d7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Killing pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Stopping container istio-proxy 42m Warning Unhealthy pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-api-a-d788ba-openshift-default-7f77dc57fc-s9h8s Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Killing pod/payload-pre-processing-e2e-api-b-02fbf5-764c8d7f95-qdnr5 Stopping container payload-pre-processing 42m Normal Killing pod/payload-processing-e2e-api-b-02fbf5-6b759bfc8-7kvrs Stopping container payload-processing 42m Normal ScalingReplicaSet deployment/e2e-api-b-bab19a-openshift-default Scaled up replica set e2e-api-b-bab19a-openshift-default-649b8d548c from 0 to 1 42m Normal Created pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Container created 42m Normal AddedLabel gateway/e2e-api-b-bab19a Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-bab19a 42m Normal Pulled pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/e2e-api-b-bab19a-openshift-default-649b8d548c Created pod: e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp 42m Normal Started pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Container started 42m Normal AddedInterface pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Add eth0 [10.134.0.180/23] from ovn-kubernetes 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-fa8dd0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-fa8dd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-bab19a Scaled up replica set payload-pre-processing-e2e-api-b-bab19a-769d9d858c from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-api-b-bab19a-665c67756 Error creating: pods "payload-processing-e2e-api-b-bab19a-665c67756-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bab19a: serviceaccount "payload-processing-e2e-api-b-bab19a" not found 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-bab19a-769d9d858c Error creating: pods "payload-pre-processing-e2e-api-b-bab19a-769d9d858c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-bab19a: serviceaccount "payload-processing-e2e-api-b-bab19a" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-bab19a Scaled up replica set payload-processing-e2e-api-b-bab19a-665c67756 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-bab19a-769d9d858c Created pod: payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc 42m Normal AddedInterface pod/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 Add eth0 [10.134.0.182/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-bab19a-665c67756 Created pod: payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc Add eth0 [10.134.0.181/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Startup probe error: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused... 42m Normal Created pod/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc Container created 42m Warning Unhealthy pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Startup probe failed: Get "http://10.134.0.168:15021/healthz/ready": dial tcp 10.134.0.168:15021: connect: connection refused 42m Normal Started pod/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc Container started 42m Normal Pulled pod/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 Container created 42m Normal Started pod/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 Container started 42m Normal Killing pod/e2e-api-b-bdc256-openshift-default-5cf7d97495-76wgr Stopping container istio-proxy 42m Normal SuccessfulCreate replicaset/e2e-inf-a-481ab7-openshift-default-59958f667c Created pod: e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w 42m Normal ScalingReplicaSet deployment/e2e-inf-a-481ab7-openshift-default Scaled up replica set e2e-inf-a-481ab7-openshift-default-59958f667c from 0 to 1 42m Normal AddedLabel gateway/e2e-inf-a-481ab7 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-481ab7 42m Normal AddedLabel gateway/e2e-inf-a-8dbe73 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-8dbe73 42m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w 42m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-tbj4w Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled down replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 6 to 5 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 5; reason: All metrics below target 42m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-x8wgq 42m Normal AddedInterface pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Add eth0 [10.134.0.183/23] from ovn-kubernetes 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 5; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled down replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 6 to 5 42m Normal SuccessfulCreate replicaset/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf Created pod: e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 42m Normal ScalingReplicaSet deployment/e2e-inf-a-8dbe73-openshift-default Scaled up replica set e2e-inf-a-8dbe73-openshift-default-6c44d95ddf from 0 to 1 42m Normal AddedInterface pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Add eth0 [10.134.0.184/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-481ab7 Scaled up replica set payload-processing-e2e-inf-a-481ab7-7fc587dfd6 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-481ab7-7fc587dfd6 Created pod: payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7 Error creating: pods "payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-481ab7: serviceaccount "payload-processing-e2e-inf-a-481ab7" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-481ab7-7fc587dfd6 Error creating: pods "payload-processing-e2e-inf-a-481ab7-7fc587dfd6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-481ab7: serviceaccount "payload-processing-e2e-inf-a-481ab7" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7 Created pod: payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-481ab7 Scaled up replica set payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7 from 0 to 1 42m Normal Started pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Container started 42m Normal Created pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Add eth0 [10.134.0.186/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Add eth0 [10.134.0.185/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Container created 42m Normal Started pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Container started 42m Normal Pulled pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9 Error creating: pods "payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-8dbe73: serviceaccount "payload-processing-e2e-inf-a-8dbe73" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-8dbe73-698f74c77d Created pod: payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-8dbe73 Scaled up replica set payload-processing-e2e-inf-a-8dbe73-698f74c77d from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-8dbe73 Scaled up replica set payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9 Created pod: payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-8dbe73-698f74c77d Error creating: pods "payload-processing-e2e-inf-a-8dbe73-698f74c77d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-8dbe73: serviceaccount "payload-processing-e2e-inf-a-8dbe73" not found 42m Normal Killing pod/payload-pre-processing-e2e-mt-52785f5e-67559cc487-mbl9d Stopping container payload-pre-processing 42m Normal Pulled pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Container started 42m Normal Started pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Container started 42m Normal Killing pod/payload-processing-e2e-mt-52785f5e-5f57cff566-5jzd5 Stopping container payload-processing 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Container created 42m Normal Created pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Container created 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Add eth0 [10.132.0.110/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Add eth0 [10.132.0.109/23] from ovn-kubernetes 42m Normal Killing pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-pgwjg Stopping container istio-proxy 42m Normal Killing pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-l5shv Stopping container istio-proxy 42m Normal Killing pod/e2e-mt-d7a6db25-openshift-default-68896bcdc7-hbqrg Stopping container istio-proxy 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-fa8dd0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-fa8dd0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Created pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk 42m Normal AddedLabel gateway/e2e-shared-a-w1-48df96 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-48df96 42m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled up replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 0 to 1 42m Normal AddedInterface pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Add eth0 [10.134.0.187/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf Error creating: pods "payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-48df96: serviceaccount "payload-processing-e2e-shared-a-w1-48df96" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-48df96 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-48df96 Scaled up replica set payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4 Created pod: payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz 41m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4 Error creating: pods "payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-48df96: serviceaccount "payload-processing-e2e-shared-a-w1-48df96" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf Created pod: payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l 41m Normal Started pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Container started 41m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Container created 41m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Add eth0 [10.134.0.189/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Container created 41m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Add eth0 [10.134.0.188/23] from ovn-kubernetes 41m Normal Started pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Container started 41m Normal Pulled pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Container created 41m Warning Unhealthy pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Startup probe failed: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Startup probe error: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused... 41m Normal Killing pod/e2e-mt-52785f5e-openshift-default-69c968b885-7cdfm Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-inf-a-51b95e-openshift-default Scaled up replica set e2e-inf-a-51b95e-openshift-default-74bf6d5cb5 from 0 to 1 41m Normal AddedLabel gateway/e2e-inf-a-51b95e Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-51b95e 41m Normal SuccessfulCreate replicaset/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5 Created pod: e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 41m Normal AddedInterface pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Add eth0 [10.132.0.112/23] from ovn-kubernetes 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt Stopping container istio-proxy 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-b3bdf9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-b3bdf9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h 41m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled down replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 10 to 6 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-qq9bt 41m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn 41m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-rc62h Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-hhkvn Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-c88jw Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-51b95e Scaled up replica set payload-processing-e2e-inf-a-51b95e-58df465794 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7 Error creating: pods "payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-51b95e: serviceaccount "payload-processing-e2e-inf-a-51b95e" not found 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Add eth0 [10.134.0.191/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-51b95e-58df465794 Error creating: pods "payload-processing-e2e-inf-a-51b95e-58df465794-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-51b95e: serviceaccount "payload-processing-e2e-inf-a-51b95e" not found 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Add eth0 [10.134.0.190/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7 Created pod: payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-51b95e Scaled up replica set payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-51b95e-58df465794 Created pod: payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl 41m Normal Started pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Container started 41m Normal Created pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Container created 41m Normal Created pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Container created 41m Normal Pulled pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Container started 41m Warning Unhealthy pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Readiness probe failed: Get "http://10.134.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Readiness probe error: Get "http://10.134.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Killing pod/e2e-api-b-02fbf5-openshift-default-64fc59d485-jhdtc Stopping container istio-proxy 41m Normal Pulled pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Container started 41m Normal Created pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Container created 41m Normal AddedLabel gateway/e2e-inf-a-8d2a70 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-8d2a70 41m Normal SuccessfulCreate replicaset/e2e-inf-a-8d2a70-openshift-default-677474fcd7 Created pod: e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg 41m Normal ScalingReplicaSet deployment/e2e-inf-a-8d2a70-openshift-default Scaled up replica set e2e-inf-a-8d2a70-openshift-default-677474fcd7 from 0 to 1 41m Normal AddedInterface pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Add eth0 [10.134.0.192/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-8d2a70 Scaled up replica set payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9 from 0 to 1 41m Warning Unhealthy pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Startup probe failed: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-8d2a70 Scaled up replica set payload-processing-e2e-inf-a-8d2a70-cd9f5c85b from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b Error creating: pods "payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-8d2a70: serviceaccount "payload-processing-e2e-inf-a-8d2a70" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9 Error creating: pods "payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-8d2a70: serviceaccount "payload-processing-e2e-inf-a-8d2a70" not found 41m Warning ProbeError pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Startup probe error: Get "http://10.134.0.180:15021/healthz/ready": dial tcp 10.134.0.180:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-b3bdf9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9 Created pod: payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Add eth0 [10.134.0.193/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Container started 41m Normal Pulled pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-b3bdf9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b Created pod: payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx 41m Normal Created pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Container created 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Add eth0 [10.134.0.194/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Container created 41m Normal Started pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Container started 41m Normal Created pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Container created 41m Normal Pulled pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Add eth0 [10.134.0.195/23] from ovn-kubernetes 41m Normal SuccessfulCreate replicaset/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9 Created pod: e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-6444d7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-inf-a-6444d7-openshift-default Scaled up replica set e2e-inf-a-6444d7-openshift-default-7bf76bdcd9 from 1 to 2 41m Warning ProbeError pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 41m Normal Killing pod/payload-pre-processing-e2e-api-a-fa8dd0-77bcf7f86c-rzx8b Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-api-a-fa8dd0-9dbb6d4dd-cbtwp Stopping container payload-processing 41m Normal AddedLabel gateway/e2e-inf-b-03d892 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-03d892 41m Normal ScalingReplicaSet deployment/e2e-inf-b-03d892-openshift-default Scaled up replica set e2e-inf-b-03d892-openshift-default-6d4cdfbfd9 from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9 Created pod: e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw 41m Normal AddedInterface pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Add eth0 [10.134.0.196/23] from ovn-kubernetes 41m Normal AddedLabel gateway/e2e-inf-b-84b89f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-84b89f 41m Normal SuccessfulCreate replicaset/e2e-inf-b-84b89f-openshift-default-f7d86d855 Created pod: e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw 41m Normal ScalingReplicaSet deployment/e2e-inf-b-84b89f-openshift-default Scaled up replica set e2e-inf-b-84b89f-openshift-default-f7d86d855 from 0 to 1 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-8d2a70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-8d2a70-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal AddedInterface pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Add eth0 [10.132.0.114/23] from ovn-kubernetes 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-03d892-696b99c46 Error creating: pods "payload-processing-e2e-inf-b-03d892-696b99c46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-03d892: serviceaccount "payload-processing-e2e-inf-b-03d892" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-03d892 Scaled up replica set payload-processing-e2e-inf-b-03d892-696b99c46 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-03d892-8674cc764c Error creating: pods "payload-pre-processing-e2e-inf-b-03d892-8674cc764c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-03d892: serviceaccount "payload-processing-e2e-inf-b-03d892" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-03d892 Scaled up replica set payload-pre-processing-e2e-inf-b-03d892-8674cc764c from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-03d892-696b99c46 Created pod: payload-processing-e2e-inf-b-03d892-696b99c46-mb79k 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-03d892-8674cc764c Created pod: payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-84b89f Scaled up replica set payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c from 0 to 1 40m Normal Created pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Container created 40m Normal Pulled pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k Container started 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-84b89f-fd65db4cb Created pod: payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 40m Normal AddedInterface pod/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k Add eth0 [10.134.0.197/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Add eth0 [10.134.0.198/23] from ovn-kubernetes 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-84b89f-fd65db4cb Error creating: pods "payload-processing-e2e-inf-b-84b89f-fd65db4cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-84b89f: serviceaccount "payload-processing-e2e-inf-b-84b89f" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-84b89f Scaled up replica set payload-processing-e2e-inf-b-84b89f-fd65db4cb from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c Error creating: pods "payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-84b89f: serviceaccount "payload-processing-e2e-inf-b-84b89f" not found 40m Normal Created pod/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k Container created 40m Normal Started pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Container started 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c Created pod: payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb 40m Normal Created pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Container created 40m Normal Started pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Container started 40m Normal AddedInterface pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Add eth0 [10.134.0.199/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Add eth0 [10.134.0.200/23] from ovn-kubernetes 40m Normal Started pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Container started 40m Normal Pulled pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Container created 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-bab19a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-bab19a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Readiness probe error: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Readiness probe failed: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Readiness probe error: Get "http://10.134.0.184:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Readiness probe failed: Get "http://10.134.0.184:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Normal Pulled pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Container started 40m Normal Created pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Container created 40m Normal Pulled pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Container created 40m Normal Started pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Container started 40m Normal Created pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Container created 40m Normal Started pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Container started 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-481ab7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Startup probe failed: Get "http://10.134.0.187:15021/healthz/ready": dial tcp 10.134.0.187:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Startup probe error: Get "http://10.134.0.187:15021/healthz/ready": dial tcp 10.134.0.187:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Startup probe error: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Startup probe failed: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Startup probe failed: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-481ab7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Startup probe error: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused... 40m Normal SuccessfulCreate replicaset/e2e-inf-b-b3bdf9-openshift-default-755bccddf Created pod: e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz 40m Normal ScalingReplicaSet deployment/e2e-inf-b-b3bdf9-openshift-default Scaled up replica set e2e-inf-b-b3bdf9-openshift-default-755bccddf from 1 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-b3bdf9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal Pulled pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Container created 40m Normal Started pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Container started 40m Normal AddedInterface pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Add eth0 [10.132.0.115/23] from ovn-kubernetes 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-8dbe73-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-8dbe73-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-cce513-openshift-default Scaled up replica set e2e-shared-b-w1-cce513-openshift-default-84cc9879 from 0 to 1 40m Normal AddedLabel gateway/e2e-shared-b-w1-cce513 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-cce513 40m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-cce513-openshift-default-84cc9879 Created pod: e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 40m Normal AddedInterface pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Add eth0 [10.134.0.201/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Startup probe failed: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Startup probe error: Get "http://10.132.0.115:15021/healthz/ready": dial tcp 10.132.0.115:15021: connect: connection refused... 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-cce513 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-cce513 Scaled up replica set payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8 from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-cce513: serviceaccount "payload-processing-e2e-shared-b-w1-cce513" not found 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59 Created pod: payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8 Created pod: payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb 40m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8 Error creating: pods "payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-cce513: serviceaccount "payload-processing-e2e-shared-b-w1-cce513" not found 40m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Container started 40m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Add eth0 [10.134.0.202/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Container started 40m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Container created 40m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Add eth0 [10.134.0.203/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Container created 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-481ab7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-481ab7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedLabel gateway/e2e-inf-b-b1f6ed Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-b1f6ed 40m Normal ScalingReplicaSet deployment/e2e-inf-b-b1f6ed-openshift-default Scaled up replica set e2e-inf-b-b1f6ed-openshift-default-767786fdd5 from 0 to 1 40m Normal SuccessfulCreate replicaset/e2e-inf-b-b1f6ed-openshift-default-767786fdd5 Created pod: e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 40m Normal AddedInterface pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Add eth0 [10.132.0.116/23] from ovn-kubernetes 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-8d2a70-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal SuccessfulCreate replicaset/e2e-inf-a-8d2a70-openshift-default-677474fcd7 Created pod: e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal ScalingReplicaSet deployment/e2e-inf-a-8d2a70-openshift-default Scaled up replica set e2e-inf-a-8d2a70-openshift-default-677474fcd7 from 1 to 4 40m Normal SuccessfulCreate replicaset/e2e-inf-a-8d2a70-openshift-default-677474fcd7 Created pod: e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x 40m Normal SuccessfulCreate replicaset/e2e-inf-a-8d2a70-openshift-default-677474fcd7 Created pod: e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw 40m Normal Started pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Container started 40m Normal Started pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Container started 40m Normal Pulled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Container created 40m Normal Pulled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Add eth0 [10.132.0.117/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Container created 40m Normal AddedInterface pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Add eth0 [10.134.0.204/23] from ovn-kubernetes 40m Normal AddedInterface pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Add eth0 [10.133.0.141/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Container created 40m Normal Started pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Container started 40m Warning Unhealthy pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Readiness probe failed: Get "http://10.134.0.192:15021/healthz/ready": read tcp 10.134.0.2:56086->10.134.0.192:15021: read: connection reset by peer 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-b1f6ed Scaled up replica set payload-processing-e2e-inf-b-b1f6ed-579f46c958 from 0 to 1 40m Warning ProbeError pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Readiness probe error: Get "http://10.134.0.192:15021/healthz/ready": read tcp 10.134.0.2:56086->10.134.0.192:15021: read: connection reset by peer... 40m Warning ProbeError pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Readiness probe error: Get "http://10.132.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Readiness probe failed: Get "http://10.132.0.112:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Normal Pulled pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Add eth0 [10.134.0.205/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c Created pod: payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c Error creating: pods "payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-b1f6ed: serviceaccount "payload-processing-e2e-inf-b-b1f6ed" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-b1f6ed Scaled up replica set payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-b1f6ed-579f46c958 Error creating: pods "payload-processing-e2e-inf-b-b1f6ed-579f46c958-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-b1f6ed: serviceaccount "payload-processing-e2e-inf-b-b1f6ed" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-b1f6ed-579f46c958 Created pod: payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j 40m Normal Started pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Container started 40m Normal Created pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Container created 40m Normal Created pod/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j Container created 40m Normal Created pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Container created 40m Normal AddedInterface pod/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j Add eth0 [10.134.0.206/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Container started 40m Normal Started pod/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j Container started 40m Normal Pulled pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Pulled pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Startup probe error: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-8dbe73-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Startup probe failed: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused 40m Normal Created pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Container created 40m Normal Started pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Container started 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-8dbe73-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 40m Normal Killing pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Stopping container istio-proxy 40m Warning ProbeError pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Readiness probe failed: HTTP probe failed with statuscode: 503 40m Warning ProbeError pod/e2e-api-a-fa8dd0-openshift-default-6588f97d75-8dzwc Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal AddedLabel gateway/e2e-inf-b-5007e1 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-5007e1 40m Normal SuccessfulCreate replicaset/e2e-inf-b-5007e1-openshift-default-64c644d8cc Created pod: e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc 40m Normal ScalingReplicaSet deployment/e2e-inf-b-5007e1-openshift-default Scaled up replica set e2e-inf-b-5007e1-openshift-default-64c644d8cc from 0 to 1 40m Normal AddedInterface pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Add eth0 [10.134.0.207/23] from ovn-kubernetes 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-5007e1 Scaled up replica set payload-processing-e2e-inf-b-5007e1-6b98658bc4 from 0 to 1 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-5007e1-6b98658bc4 Created pod: payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-5007e1 Scaled up replica set payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-5007e1-6b98658bc4 Error creating: pods "payload-processing-e2e-inf-b-5007e1-6b98658bc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-5007e1: serviceaccount "payload-processing-e2e-inf-b-5007e1" not found 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c Created pod: payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c Error creating: pods "payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-5007e1: serviceaccount "payload-processing-e2e-inf-b-5007e1" not found 39m Normal Created pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Container created 39m Normal AddedInterface pod/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn Add eth0 [10.134.0.208/23] from ovn-kubernetes 39m Normal Started pod/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn Container started 39m Normal Created pod/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn Container created 39m Normal Pulled pod/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Killing pod/payload-pre-processing-e2e-api-b-bab19a-769d9d858c-gq8fc Stopping container payload-pre-processing 39m Normal Pulled pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Container started 39m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Add eth0 [10.132.0.119/23] from ovn-kubernetes 39m Normal Killing pod/payload-processing-e2e-api-b-bab19a-665c67756-zz5k5 Stopping container payload-processing 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-8d2a70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-8d2a70-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning ProbeError pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Readiness probe error: Get "http://10.134.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Readiness probe failed: Get "http://10.134.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-51b95e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-51b95e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Created pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Container created 39m Normal Started pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Container started 39m Normal Pulled pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Warning ProbeError pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Startup probe error: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Startup probe failed: Get "http://10.134.0.196:15021/healthz/ready": dial tcp 10.134.0.196:15021: connect: connection refused 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-03d892-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-03d892-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Deleted pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 39m Normal Killing pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-9cjs5 Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-bmwfx 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tlpfp 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 4; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-znxq4 Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default New size: 3; reason: All metrics below target 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 4; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default Scaled down replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 5 to 4 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 6; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled down replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 4 to 3 39m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled down replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 8 to 6 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 39m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-ztgc9 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled down replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 5 to 4 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled down replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 5 to 4 39m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-7rsqp Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled down replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 6 to 4 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 4; reason: All metrics below target 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 4; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-2h2n6 Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-s5rzw Stopping container istio-proxy 39m Normal Pulled pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Container started 39m Normal Created pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Container created 39m Warning ProbeError pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-84b89f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-84b89f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled down replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 6 to 5 38m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 Stopping container istio-proxy 38m Normal Started pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Container started 38m Normal Created pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Container created 38m Normal Pulled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 5; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-t4s54 38m Normal Created pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Container created 38m Normal Started pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Container started 38m Normal Pulled pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-03d892-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning Unhealthy pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 38m Warning ProbeError pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-03d892-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-b1f6ed-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-b1f6ed-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning Unhealthy pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal Killing pod/e2e-api-b-bab19a-openshift-default-649b8d548c-hj4gp Stopping container istio-proxy 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-b1f6ed-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-b1f6ed-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal AddedLabel gateway/e2e-inf-a-ef1084 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-ef1084 38m Normal SuccessfulCreate replicaset/e2e-inf-a-ef1084-openshift-default-68989c6b9f Created pod: e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq 38m Normal ScalingReplicaSet deployment/e2e-inf-a-ef1084-openshift-default Scaled up replica set e2e-inf-a-ef1084-openshift-default-68989c6b9f from 0 to 1 38m Normal AddedInterface pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Add eth0 [10.132.0.141/23] from ovn-kubernetes 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff Created pod: payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff Error creating: pods "payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-ef1084: serviceaccount "payload-processing-e2e-inf-a-ef1084" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-ef1084 Scaled up replica set payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-ef1084 Scaled up replica set payload-processing-e2e-inf-a-ef1084-66c68bf584 from 0 to 1 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-ef1084-66c68bf584 Error creating: pods "payload-processing-e2e-inf-a-ef1084-66c68bf584-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-ef1084: serviceaccount "payload-processing-e2e-inf-a-ef1084" not found 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-ef1084-66c68bf584 Created pod: payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z 38m Normal Pulled pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Container started 38m Normal Created pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Container created 38m Normal Pulled pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Add eth0 [10.132.0.144/23] from ovn-kubernetes 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Add eth0 [10.132.0.143/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Container started 38m Normal Started pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Container started 38m Normal Pulled pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Container created 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-ef1084-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal SuccessfulCreate replicaset/e2e-inf-b-84b89f-openshift-default-f7d86d855 Created pod: e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-84b89f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal ScalingReplicaSet deployment/e2e-inf-b-84b89f-openshift-default Scaled up replica set e2e-inf-b-84b89f-openshift-default-f7d86d855 from 1 to 2 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-ef1084-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Normal AddedInterface pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Add eth0 [10.134.0.221/23] from ovn-kubernetes 38m Normal Started pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Container started 38m Normal Pulled pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Container created 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-5007e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-5007e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning Unhealthy pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Startup probe failed: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Startup probe error: Get "http://10.134.0.207:15021/healthz/ready": dial tcp 10.134.0.207:15021: connect: connection refused... 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-8dbe73-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Warning Unhealthy pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Startup probe failed: Get "http://10.134.0.221:15021/healthz/ready": dial tcp 10.134.0.221:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/e2e-inf-a-8dbe73-openshift-default Scaled up replica set e2e-inf-a-8dbe73-openshift-default-6c44d95ddf from 1 to 2 37m Warning ProbeError pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Startup probe error: Get "http://10.134.0.221:15021/healthz/ready": dial tcp 10.134.0.221:15021: connect: connection refused... 37m Normal Created pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Container created 37m Normal Pulled pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal SuccessfulCreate replicaset/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf Created pod: e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q 37m Normal Started pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Container started 37m Normal AddedInterface pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Add eth0 [10.132.0.146/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/e2e-inf-b-03d892-openshift-default Scaled up replica set e2e-inf-b-03d892-openshift-default-6d4cdfbfd9 from 1 to 2 37m Normal SuccessfulCreate replicaset/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9 Created pod: e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-03d892-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal AddedInterface pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Add eth0 [10.132.0.147/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Container created 37m Normal Started pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Container started 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled up replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 1 to 2 37m Warning Unhealthy pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 10 to 8 37m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-jwx9q Stopping container istio-proxy 37m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-fwrpg Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Created pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-fwrpg 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-jwx9q 37m Warning ProbeError pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 37m Normal Pulled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Add eth0 [10.132.0.148/23] from ovn-kubernetes 37m Normal Created pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Container created 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-5007e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-5007e1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Started pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Container started 37m Warning ProbeError pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused 37m Warning ProbeError pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 37m Normal Killing pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Stopping container payload-processing 37m Normal Killing pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Stopping container payload-pre-processing 37m Normal SuccessfulCreate replicaset/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5 Created pod: e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-51b95e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal ScalingReplicaSet deployment/e2e-inf-a-51b95e-openshift-default Scaled up replica set e2e-inf-a-51b95e-openshift-default-74bf6d5cb5 from 1 to 2 37m Normal Started pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Container started 37m Normal Created pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Container created 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-6444d7-86ddb5b999-ppd6n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal AddedInterface pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Add eth0 [10.134.0.223/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-6444d7-767fdb56d6-mc4fh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning ProbeError pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Startup probe error: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Startup probe failed: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused 37m Normal Killing pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-6nnq2 Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-a-6444d7-openshift-default-7bf76bdcd9-qtjxw Stopping container istio-proxy 36m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled down replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 6 to 5 36m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h 36m Normal Killing pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Stopping container payload-processing 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 5; reason: All metrics below target 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Stopping container payload-pre-processing 36m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-d2l9h Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning Unhealthy pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Readiness probe failed: Get "http://10.132.0.141:15021/healthz/ready": read tcp 10.132.0.2:51490->10.132.0.141:15021: read: connection reset by peer 36m Warning ProbeError pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Readiness probe error: Get "http://10.132.0.141:15021/healthz/ready": read tcp 10.132.0.2:51490->10.132.0.141:15021: read: connection reset by peer... 36m Normal Started pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Container started 36m Normal Created pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Container created 36m Normal Pulled pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled down replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 5 to 4 36m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 4; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-gvb85 36m Normal ScalingReplicaSet deployment/e2e-inf-b-19bc68-openshift-default Scaled up replica set e2e-inf-b-19bc68-openshift-default-7b6485f77f from 0 to 1 36m Normal AddedLabel gateway/e2e-inf-b-19bc68 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-19bc68 36m Normal AddedInterface pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Add eth0 [10.134.0.224/23] from ovn-kubernetes 36m Normal SuccessfulCreate replicaset/e2e-inf-b-19bc68-openshift-default-7b6485f77f Created pod: e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw 36m Warning Unhealthy pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Startup probe failed: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Startup probe error: Get "http://10.132.0.141:15021/healthz/ready": dial tcp 10.132.0.141:15021: connect: connection refused... 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-cce513-openshift-default-84cc9879 Created pod: e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-cce513-openshift-default Scaled up replica set e2e-shared-b-w1-cce513-openshift-default-84cc9879 from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal AddedInterface pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Add eth0 [10.132.0.151/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-19bc68-54864556d8 Error creating: pods "payload-processing-e2e-inf-b-19bc68-54864556d8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-19bc68: serviceaccount "payload-processing-e2e-inf-b-19bc68" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-19bc68 Scaled up replica set payload-processing-e2e-inf-b-19bc68-54864556d8 from 0 to 1 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-19bc68 Scaled up replica set payload-pre-processing-e2e-inf-b-19bc68-59cc54d647 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-19bc68-54864556d8 Created pod: payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 36m Normal Pulled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Container started 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647 Error creating: pods "payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-19bc68: serviceaccount "payload-processing-e2e-inf-b-19bc68" not found 36m Normal Created pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Container created 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-b3bdf9-867787698c-vlrxj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647 Created pod: payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf 36m Warning Unhealthy pod/payload-processing-e2e-inf-b-b3bdf9-6976bd6ff8-sn6nt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 Add eth0 [10.132.0.152/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 Container created 36m Normal Created pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Container created 36m Normal Started pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Container started 36m Normal Started pod/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Add eth0 [10.132.0.153/23] from ovn-kubernetes 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-5007e1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal ScalingReplicaSet deployment/e2e-inf-b-5007e1-openshift-default Scaled up replica set e2e-inf-b-5007e1-openshift-default-64c644d8cc from 1 to 2 36m Normal SuccessfulCreate replicaset/e2e-inf-b-5007e1-openshift-default-64c644d8cc Created pod: e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 36m Normal AddedInterface pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Add eth0 [10.132.0.154/23] from ovn-kubernetes 36m Normal Started pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Container started 36m Normal Created pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Container created 36m Normal Pulled pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Startup probe failed: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused 36m Warning ProbeError pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Startup probe error: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Startup probe error: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Startup probe failed: Get "http://10.132.0.154:15021/healthz/ready": dial tcp 10.132.0.154:15021: connect: connection refused 36m Normal SuccessfulCreate replicaset/e2e-inf-b-b1f6ed-openshift-default-767786fdd5 Created pod: e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl 36m Normal ScalingReplicaSet deployment/e2e-inf-b-b1f6ed-openshift-default Scaled up replica set e2e-inf-b-b1f6ed-openshift-default-767786fdd5 from 1 to 2 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-b1f6ed-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal AddedInterface pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Add eth0 [10.134.0.225/23] from ovn-kubernetes 36m Normal Started pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Container started 36m Normal Created pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Container created 36m Normal Pulled pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-ef1084-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-ef1084-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal Killing pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cz2h7 Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-b3bdf9-openshift-default-755bccddf-cb9sz Stopping container istio-proxy 36m Normal AddedLabel gateway/e2e-shared-a-w4-beeba4 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-beeba4 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-beeba4-openshift-default Scaled up replica set e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 Created pod: e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk 36m Normal AddedInterface pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Add eth0 [10.134.0.226/23] from ovn-kubernetes 36m Warning ProbeError pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Startup probe error: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Startup probe failed: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f Error creating: pods "payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-beeba4: serviceaccount "payload-processing-e2e-shared-a-w4-beeba4" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-beeba4 Scaled up replica set payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566 Error creating: pods "payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-beeba4: serviceaccount "payload-processing-e2e-shared-a-w4-beeba4" not found 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-beeba4 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f Created pod: payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566 Created pod: payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh 36m Normal Started pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Container started 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Add eth0 [10.134.0.227/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Add eth0 [10.134.0.228/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Container started 36m Normal Created pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Container created 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Container created 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-jdlt8 35m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-jdlt8 Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 8 to 7 35m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 4; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn 35m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-kcwdn Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 3; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default Scaled down replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 4 to 3 35m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz 35m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 3; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-lhfxq 35m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-fjlsm 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default New size: 3; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default Scaled down replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 4 to 3 35m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg 35m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default Scaled down replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 4 to 3 35m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-4mggz Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 3; reason: All metrics below target 35m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Stopping container istio-proxy 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Stopping container payload-processing 35m Warning ProbeError pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-vqjxg Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-481ab7-7fc587dfd6-lcrb9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-481ab7-6c7d8bfdb7-b5zsl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Killing pod/e2e-inf-a-481ab7-openshift-default-59958f667c-kjc7w Stopping container istio-proxy 35m Normal Pulled pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Container created 35m Normal Started pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Container started 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Stopping container payload-processing 35m Warning Unhealthy pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Startup probe failed: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Startup probe error: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused... 35m Normal Killing pod/payload-processing-e2e-inf-b-03d892-696b99c46-mb79k Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Stopping container payload-pre-processing 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-8dbe73-698f74c77d-l7tpx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-8dbe73-788cc7d8c9-qnxcz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-19bc68-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-19bc68-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-03d892-8674cc764c-gtl5f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal SuccessfulDelete replicaset/e2e-worker-w6-021d6d-openshift-default-65c58b6947 Deleted pod: e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm 34m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-bpwvm Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w3-cefb27-openshift-default-577b649d97 Deleted pod: e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 34m Normal Killing pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-tgjj8 Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-worker-w6-021d6d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-021d6d-openshift-default-65c58b6947 from 3 to 2 34m Normal ScalingReplicaSet deployment/e2e-worker-w3-cefb27-openshift-default Scaled down replica set e2e-worker-w3-cefb27-openshift-default-577b649d97 from 3 to 2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-cefb27-openshift-default New size: 2; reason: All metrics below target 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-021d6d-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-19bc68-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-19bc68-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal AddedLabel gateway/e2e-shared-b-w4-73435a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-73435a 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-73435a-openshift-default Scaled up replica set e2e-shared-b-w4-73435a-openshift-default-fc454477f from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-73435a-openshift-default-fc454477f Created pod: e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b 34m Normal AddedInterface pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Add eth0 [10.132.0.166/23] from ovn-kubernetes 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-73435a Scaled up replica set payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64 Error creating: pods "payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-73435a: serviceaccount "payload-processing-e2e-shared-b-w4-73435a" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64 Created pod: payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975 Created pod: payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-73435a: serviceaccount "payload-processing-e2e-shared-b-w4-73435a" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-73435a Scaled up replica set payload-processing-e2e-shared-b-w4-73435a-7645c9bc64 from 0 to 1 34m Normal Killing pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-5hz6q Stopping container istio-proxy 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Container started 34m Normal Killing pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Stopping container istio-proxy 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Add eth0 [10.132.0.169/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Container created 34m Normal Pulled pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Add eth0 [10.132.0.168/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Container created 34m Normal Started pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Container started 34m Warning Unhealthy pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-inf-a-8dbe73-openshift-default-6c44d95ddf-cdcq6 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq 34m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-tkdxq Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-f5kkw Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-03d892-openshift-default-6d4cdfbfd9-qs6xc Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-009e78-openshift-default Scaled up replica set e2e-shared-a-w6-009e78-openshift-default-5674f58c7 from 0 to 1 34m Normal AddedLabel gateway/e2e-shared-a-w6-009e78 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-009e78 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-009e78-openshift-default-5674f58c7 Created pod: e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w 34m Normal AddedInterface pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Add eth0 [10.134.0.229/23] from ovn-kubernetes 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Stopping container payload-processing 34m Normal Created pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Container created 34m Normal ScalingReplicaSet deployment/e2e-inf-a-ef1084-openshift-default Scaled up replica set e2e-inf-a-ef1084-openshift-default-68989c6b9f from 1 to 3 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-ef1084-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 34m Normal SuccessfulCreate replicaset/e2e-inf-a-ef1084-openshift-default-68989c6b9f Created pod: e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 34m Normal SuccessfulCreate replicaset/e2e-inf-a-ef1084-openshift-default-68989c6b9f Created pod: e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn 34m Normal AddedInterface pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Add eth0 [10.134.0.230/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Container started 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled up replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 2 to 3 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Created pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh 34m Normal Pulled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Add eth0 [10.133.0.149/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Add eth0 [10.133.0.150/23] from ovn-kubernetes 34m Normal Created pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Container created 34m Normal Started pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Container started 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 34m Normal Created pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Container created 34m Normal Started pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Container started 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-009e78 Scaled up replica set payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d Error creating: pods "payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-009e78: serviceaccount "payload-processing-e2e-shared-a-w6-009e78" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9 Created pod: payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9 Error creating: pods "payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-009e78: serviceaccount "payload-processing-e2e-shared-a-w6-009e78" not found 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-009e78 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9 from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d Created pod: payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m 34m Normal Created pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Container created 34m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Container started 34m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Container created 34m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Add eth0 [10.132.0.173/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Container started 34m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Add eth0 [10.132.0.172/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Startup probe failed: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Startup probe error: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Startup probe failed: Get "http://10.133.0.150:15021/healthz/ready": dial tcp 10.133.0.150:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Startup probe error: Get "http://10.133.0.150:15021/healthz/ready": dial tcp 10.133.0.150:15021: connect: connection refused... 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-cce513-openshift-default-84cc9879 Created pod: e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-cce513-openshift-default-84cc9879 Created pod: e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-cce513-openshift-default Scaled up replica set e2e-shared-b-w1-cce513-openshift-default-84cc9879 from 2 to 4 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal AddedInterface pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Add eth0 [10.133.0.151/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-b1f6ed-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 34m Normal Pulled pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Add eth0 [10.132.0.176/23] from ovn-kubernetes 34m Normal ScalingReplicaSet deployment/e2e-inf-b-b1f6ed-openshift-default Scaled up replica set e2e-inf-b-b1f6ed-openshift-default-767786fdd5 from 2 to 3 34m Normal Started pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Container started 34m Normal Started pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Container started 34m Normal Created pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Container created 34m Normal Created pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Container created 34m Normal Started pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Container started 34m Normal Created pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Container created 34m Normal Pulled pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal SuccessfulCreate replicaset/e2e-inf-b-b1f6ed-openshift-default-767786fdd5 Created pod: e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d 34m Normal AddedInterface pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Add eth0 [10.133.0.152/23] from ovn-kubernetes 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Stopping container payload-processing 33m Warning Unhealthy pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-84b89f-c9c9df96c-7rfbb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-inf-b-84b89f-fd65db4cb-4rpk4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-inf-a-8d2a70-cd9f5c85b-k28lx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-8d2a70-7bc7f566f9-rq746 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Killing pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-jldws Stopping container istio-proxy 33m Warning ProbeError pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-b-84b89f-openshift-default-f7d86d855-mddrw Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal Killing pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-4989x Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-899lw Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-vpb7n Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-a-8d2a70-openshift-default-677474fcd7-5kctg Readiness probe error: HTTP probe failed with statuscode: 503... 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-inf-b-5007e1-6b98658bc4-8x5fn Stopping container payload-processing 33m Normal Pulled pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal SuccessfulCreate replicaset/e2e-inf-b-19bc68-openshift-default-7b6485f77f Created pod: e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw 33m Normal ScalingReplicaSet deployment/e2e-inf-b-19bc68-openshift-default Scaled up replica set e2e-inf-b-19bc68-openshift-default-7b6485f77f from 1 to 2 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-19bc68-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal Created pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Container created 33m Normal Started pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Container started 33m Normal Created pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Container created 33m Normal Pulled pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Add eth0 [10.132.0.181/23] from ovn-kubernetes 33m Normal Started pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Container started 33m Normal Killing pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Stopping container payload-pre-processing 33m Normal Killing pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Stopping container payload-processing 33m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 7 to 5 33m Warning ProbeError pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-kfpcv 33m Warning Unhealthy pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-xx8n6 33m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-xx8n6 Stopping container istio-proxy 33m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-kfpcv Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Startup probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Created pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled up replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 3 to 5 33m Warning ProbeError pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Startup probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 33m Normal AddedInterface pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Add eth0 [10.134.0.232/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Created pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 33m Normal Pulled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Pulled pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Container created 33m Normal Started pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Container started 33m Normal AddedInterface pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Add eth0 [10.132.0.182/23] from ovn-kubernetes 33m Normal Started pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Container started 33m Normal Created pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Container created 32m Normal Killing pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-swcm7 Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Stopping container istio-proxy 32m Normal SuccessfulDelete replicaset/e2e-worker-w4-ca5363-openshift-default-6cfd85d975 Deleted pod: e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 3; reason: All metrics below target 32m Warning ProbeError pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Startup probe error: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused... 32m Normal ScalingReplicaSet deployment/e2e-worker-w4-ca5363-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-ca5363-openshift-default-6cfd85d975 from 3 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-ca5363-openshift-default New size: 2; reason: All metrics below target 32m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x Stopping container istio-proxy 32m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-vzqgg Stopping container istio-proxy 32m Warning Unhealthy pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Startup probe failed: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused 32m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-4fz6x 32m Warning ProbeError pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-a-51b95e-openshift-default-74bf6d5cb5-tnnfl Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-5007e1-d7bb8bb4c-xbqmg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-51b95e-56d4b8b8b7-9fb4d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-processing-e2e-inf-a-51b95e-58df465794-v4nvl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-7f92f6-openshift-default Scaled up replica set e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 from 0 to 1 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Created pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh 32m Normal AddedLabel gateway/e2e-shared-b-w6-7f92f6 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-7f92f6 32m Normal Killing pod/payload-processing-e2e-inf-b-b1f6ed-579f46c958-7cc8j Stopping container payload-processing 32m Normal Killing pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Stopping container payload-pre-processing 32m Normal AddedInterface pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Add eth0 [10.134.0.233/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-7f92f6-5744474458 Created pod: payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-7f92f6 Scaled up replica set payload-processing-e2e-shared-b-w6-7f92f6-5744474458 from 0 to 1 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-7f92f6-5744474458 Error creating: pods "payload-processing-e2e-shared-b-w6-7f92f6-5744474458-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-7f92f6: serviceaccount "payload-processing-e2e-shared-b-w6-7f92f6" not found 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c Error creating: pods "payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-7f92f6: serviceaccount "payload-processing-e2e-shared-b-w6-7f92f6" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-7f92f6 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c Created pod: payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp Add eth0 [10.133.0.156/23] from ovn-kubernetes 32m Normal Started pod/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl Container started 32m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp Container created 32m Normal Pulled pod/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl Container created 32m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp Container started 32m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl Add eth0 [10.133.0.157/23] from ovn-kubernetes 32m Normal Killing pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-xzwxc Stopping container istio-proxy 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning ProbeError pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-b-5007e1-openshift-default-64c644d8cc-wrs68 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-b1f6ed-6d7477cd6c-cfgvb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Started pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Container started 32m Normal Created pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Container created 32m Normal Pulled pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Killing pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-5gvj4 Stopping container istio-proxy 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 Created pod: e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-93079c-openshift-default Scaled up replica set e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 from 0 to 1 32m Normal AddedLabel gateway/e2e-shared-a-w5-93079c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-93079c 32m Normal AddedInterface pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Add eth0 [10.134.0.234/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Startup probe failed: Get "http://10.134.0.229:15021/healthz/ready": dial tcp 10.134.0.229:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Startup probe error: Get "http://10.134.0.229:15021/healthz/ready": dial tcp 10.134.0.229:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning ProbeError pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-g62bl Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning Unhealthy pod/e2e-inf-b-b1f6ed-openshift-default-767786fdd5-ghc4d Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-93079c Scaled up replica set payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7 from 0 to 1 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-93079c Scaled up replica set payload-processing-e2e-shared-a-w5-93079c-668d8c594d from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-93079c-668d8c594d Created pod: payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-93079c: serviceaccount "payload-processing-e2e-shared-a-w5-93079c" not found 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-93079c-668d8c594d Error creating: pods "payload-processing-e2e-shared-a-w5-93079c-668d8c594d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-93079c: serviceaccount "payload-processing-e2e-shared-a-w5-93079c" not found 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 Add eth0 [10.134.0.235/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7 Created pod: payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 32m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 Container started 32m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 Container created 32m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p Add eth0 [10.134.0.236/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p Container created 32m Normal Pulled pod/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p Container started 31m Normal Started pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Container started 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-11f521-openshift-default Scaled up replica set e2e-shared-a-w3-11f521-openshift-default-96689b67c from 1 to 2 31m Normal Created pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Container created 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-11f521-openshift-default-96689b67c Created pod: e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg 31m Normal Pulled pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Add eth0 [10.132.0.185/23] from ovn-kubernetes 31m Warning Unhealthy pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Created pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Created pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Created pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-7f92f6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-7f92f6-openshift-default Scaled up replica set e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 from 1 to 4 31m Normal Created pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Container created 31m Normal Pulled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Add eth0 [10.134.0.238/23] from ovn-kubernetes 31m Normal Started pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Container started 31m Normal Created pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Container created 31m Normal Pulled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Add eth0 [10.133.0.164/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Container created 31m Normal Started pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Container started 31m Normal Started pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Container started 31m Normal AddedInterface pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Add eth0 [10.132.0.186/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Container created 31m Normal Started pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Container started 31m Normal Pulled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Container created 31m Warning ProbeError pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Readiness probe error: Get "http://10.134.0.233:15021/healthz/ready": read tcp 10.134.0.2:59842->10.134.0.233:15021: read: connection reset by peer... 31m Warning Unhealthy pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Readiness probe failed: Get "http://10.134.0.233:15021/healthz/ready": read tcp 10.134.0.2:59842->10.134.0.233:15021: read: connection reset by peer 31m Normal Pulled pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Container started 31m Warning Unhealthy pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Startup probe error: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Startup probe failed: Get "http://10.133.0.164:15021/healthz/ready": dial tcp 10.133.0.164:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Startup probe failed: Get "http://10.134.0.238:15021/healthz/ready": dial tcp 10.134.0.238:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Startup probe error: Get "http://10.134.0.238:15021/healthz/ready": dial tcp 10.134.0.238:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Startup probe error: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Startup probe failed: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Startup probe error: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Startup probe failed: Get "http://10.134.0.233:15021/healthz/ready": dial tcp 10.134.0.233:15021: connect: connection refused 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59 Created pod: e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-5f387c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5f387c-openshift-default Scaled up replica set e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59 from 1 to 2 31m Normal Pulled pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Container created 31m Normal AddedInterface pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Add eth0 [10.132.0.188/23] from ovn-kubernetes 31m Normal Started pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Container started 31m Warning Unhealthy pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 31m Normal Killing pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Stopping container payload-pre-processing 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-7f92f6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-7f92f6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-beeba4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-beeba4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal AddedLabel gateway/e2e-shared-b-w5-a5e9ab Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-a5e9ab 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a5e9ab-openshift-default Scaled up replica set e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9 from 0 to 1 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9 Created pod: e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh 30m Normal AddedInterface pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Add eth0 [10.134.0.239/23] from ovn-kubernetes 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-ef1084-66c68bf584-58x9z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-ef1084-64c6bbc5ff-rrjq8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-7f92f6-openshift-default Scaled up replica set e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 from 4 to 5 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-7f92f6-openshift-default New size: 5; reason: All metrics below target 30m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Created pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 30m Normal Started pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Container started 30m Normal AddedInterface pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Add eth0 [10.132.0.191/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Container created 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-a5e9ab Scaled up replica set payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9 from 0 to 1 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-a5e9ab Scaled up replica set payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974 from 0 to 1 30m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974 Error creating: pods "payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-a5e9ab: serviceaccount "payload-processing-e2e-shared-b-w5-a5e9ab" not found 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 Created pod: e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 Created pod: e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 30m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Add eth0 [10.134.0.240/23] from ovn-kubernetes 30m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-a5e9ab: serviceaccount "payload-processing-e2e-shared-b-w5-a5e9ab" not found 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974 Created pod: payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9 Created pod: payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-93079c-openshift-default Scaled up replica set e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 from 1 to 3 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-93079c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal Created pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Container created 30m Normal Started pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Container started 30m Normal Created pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Container created 30m Normal Started pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Container started 30m Normal AddedInterface pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Add eth0 [10.133.0.165/23] from ovn-kubernetes 30m Normal Pulled pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Container started 30m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Container created 30m Normal Pulled pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Add eth0 [10.134.0.241/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Add eth0 [10.132.0.193/23] from ovn-kubernetes 30m Warning ProbeError pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Readiness probe error: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Readiness probe failed: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused 30m Normal Pulled pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Container created 30m Normal Started pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Container started 30m Normal Created pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Container created 30m Normal Pulled pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Container started 30m Warning ProbeError pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Startup probe error: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Startup probe failed: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Startup probe failed: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Startup probe error: Get "http://10.132.0.193:15021/healthz/ready": dial tcp 10.132.0.193:15021: connect: connection refused... 30m Normal SuccessfulCreate replicaset/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7 Created pod: e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 30m Normal AddedLabel gateway/e2e-ipp-a-55a4c7 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-55a4c7 30m Normal ScalingReplicaSet deployment/e2e-ipp-a-55a4c7-openshift-default Scaled up replica set e2e-ipp-a-55a4c7-openshift-default-759c67ccc7 from 0 to 1 30m Normal AddedInterface pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Add eth0 [10.134.0.242/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Startup probe failed: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Startup probe error: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused... 30m Normal Killing pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-lrmtq Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-bh9g4 Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-ef1084-openshift-default-68989c6b9f-jmgcn Stopping container istio-proxy 30m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-55a4c7 Scaled up replica set payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc Created pod: payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 30m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-55a4c7 Scaled up replica set payload-processing-e2e-ipp-a-55a4c7-9c65778db from 0 to 1 30m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-55a4c7-9c65778db Created pod: payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h 30m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-55a4c7-9c65778db Error creating: pods "payload-processing-e2e-ipp-a-55a4c7-9c65778db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-55a4c7: serviceaccount "payload-processing-e2e-ipp-a-55a4c7" not found 30m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc Error creating: pods "payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-55a4c7: serviceaccount "payload-processing-e2e-ipp-a-55a4c7" not found 30m Normal AddedInterface pod/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h Add eth0 [10.134.0.243/23] from ovn-kubernetes 30m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 Add eth0 [10.134.0.244/23] from ovn-kubernetes 30m Normal Started pod/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h Container started 30m Normal Created pod/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 Container created 30m Normal Started pod/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 Container started 30m Normal Pulled pod/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 30m Normal Created pod/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h Container created 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-19bc68-54864556d8-n2648 Stopping container payload-processing 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-93079c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-93079c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-19bc68-59cc54d647-zvgzf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal ScalingReplicaSet deployment/e2e-ipp-a-73f667-openshift-default Scaled up replica set e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9 from 0 to 1 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9 Created pod: e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 29m Normal AddedLabel gateway/e2e-ipp-a-73f667 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-73f667 29m Normal AddedInterface pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Add eth0 [10.132.0.196/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-73f667 Scaled up replica set payload-pre-processing-e2e-ipp-a-73f667-5f44887555 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-73f667 Scaled up replica set payload-processing-e2e-ipp-a-73f667-64d5696dc4 from 0 to 1 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j Container created 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-73f667-5f44887555 Error creating: pods "payload-pre-processing-e2e-ipp-a-73f667-5f44887555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f667: serviceaccount "payload-processing-e2e-ipp-a-73f667" not found 29m Normal Created pod/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq Container created 29m Normal Started pod/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq Container started 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq Add eth0 [10.134.0.245/23] from ovn-kubernetes 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-73f667-64d5696dc4 Error creating: pods "payload-processing-e2e-ipp-a-73f667-64d5696dc4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f667: serviceaccount "payload-processing-e2e-ipp-a-73f667" not found 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-73f667-5f44887555 Created pod: payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-73f667-64d5696dc4 Created pod: payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j Container started 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j Add eth0 [10.134.0.246/23] from ovn-kubernetes 29m Normal Pulled pod/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Killing pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-tz5sw Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-009e78-openshift-default Scaled up replica set e2e-shared-a-w6-009e78-openshift-default-5674f58c7 from 1 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-009e78-openshift-default-5674f58c7 Created pod: e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9 Created pod: e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-f64dc0-openshift-default Scaled up replica set e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9 from 0 to 1 29m Normal AddedLabel gateway/e2e-shared-a-w2-f64dc0 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-f64dc0 29m Normal AddedInterface pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Add eth0 [10.134.0.247/23] from ovn-kubernetes 29m Normal Started pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Container started 29m Normal Created pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Container created 29m Normal Pulled pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Add eth0 [10.132.0.197/23] from ovn-kubernetes 29m Warning Unhealthy pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Readiness probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 29m Warning ProbeError pod/e2e-inf-b-19bc68-openshift-default-7b6485f77f-s2xkw Readiness probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-f64dc0 Scaled up replica set payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d from 0 to 1 29m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d Error creating: pods "payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-f64dc0: serviceaccount "payload-processing-e2e-shared-a-w2-f64dc0" not found 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-f64dc0 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899 from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899 Created pod: payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-f64dc0: serviceaccount "payload-processing-e2e-shared-a-w2-f64dc0" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d Created pod: payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp 29m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Container created 29m Normal Started pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Container started 29m Normal Created pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Container created 29m Normal Pulled pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Add eth0 [10.134.0.248/23] from ovn-kubernetes 29m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Container started 29m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Add eth0 [10.132.0.198/23] from ovn-kubernetes 29m Warning ProbeError pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-73f667-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-73f667-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning ProbeError pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Readiness probe error: Get "http://10.134.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Warning Unhealthy pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Readiness probe failed: Get "http://10.134.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning Unhealthy pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Readiness probe failed: Get "http://10.134.0.242:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Warning ProbeError pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Readiness probe error: Get "http://10.134.0.242:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Normal Started pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Container started 29m Normal Pulled pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Container created 29m Normal Started pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Container started 29m Normal Created pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Container created 29m Normal Pulled pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default Scaled down replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 3 to 2 29m Warning Unhealthy pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Startup probe failed: Get "http://10.134.0.239:15021/healthz/ready": dial tcp 10.134.0.239:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Startup probe error: Get "http://10.134.0.239:15021/healthz/ready": dial tcp 10.134.0.239:15021: connect: connection refused... 29m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-gwlz9 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default New size: 2; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w0-52650f-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-52650f-openshift-default-6f5fdf664f from 3 to 2 29m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 3 to 2 29m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w0-52650f-openshift-default-6f5fdf664f Deleted pod: e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp 29m Warning Unhealthy pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Startup probe failed: Get "http://10.134.0.242:15021/healthz/ready": dial tcp 10.134.0.242:15021: connect: connection refused 29m Warning ProbeError pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Startup probe error: Get "http://10.134.0.242:15021/healthz/ready": dial tcp 10.134.0.242:15021: connect: connection refused... 29m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-worker-w2-1a11cf-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-1a11cf-openshift-default-748ddd859 from 3 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-1a11cf-openshift-default New size: 2; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w2-1a11cf-openshift-default-748ddd859 Deleted pod: e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb 29m Normal SuccessfulDelete replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Deleted pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-j9nfb 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-52650f-openshift-default New size: 2; reason: All metrics below target 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-5qfmb Stopping container istio-proxy 29m Normal Killing pod/payload-processing-e2e-ipp-a-55a4c7-9c65778db-9wf5h Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-55a4c7-d778575bc-rn5m6 Stopping container payload-pre-processing 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-a5e9ab-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-a5e9ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 29m Warning ProbeError pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-hsssp Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 29m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-5gbdq Stopping container discovery 29m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-5gbdq 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-kxdph 29m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7d9897b54c from 5 to 4 29m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-kxdph Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-11f521-openshift-default-96689b67c Created pod: e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-11f521-openshift-default Scaled up replica set e2e-shared-a-w3-11f521-openshift-default-96689b67c from 2 to 3 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 28m Normal Pulled pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Container created 28m Normal Started pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Container started 28m Normal AddedInterface pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Add eth0 [10.133.0.172/23] from ovn-kubernetes 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-55a4c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-55a4c7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning ProbeError pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-a5e9ab-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-a5e9ab-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-55a4c7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-55a4c7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Created pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Container created 28m Normal Pulled pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Container started 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-55a4c7-85555b48dd Error creating: pods "payload-processing-e2e-ipp-a-55a4c7-85555b48dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-55a4c7: serviceaccount "payload-processing-e2e-ipp-a-55a4c7" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-55a4c7-85555b48dd Created pod: payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-55a4c7 Scaled up replica set payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc Created pod: payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-55a4c7 Scaled up replica set payload-processing-e2e-ipp-a-55a4c7-85555b48dd from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc Error creating: pods "payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-55a4c7: serviceaccount "payload-processing-e2e-ipp-a-55a4c7" not found 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Add eth0 [10.134.0.250/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Add eth0 [10.134.0.249/23] from ovn-kubernetes 28m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal Pulling pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-beeba4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-beeba4-openshift-default Scaled up replica set e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 from 1 to 2 28m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 Created pod: e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Container created 28m Normal Started pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Container started 28m Normal Started pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Container started 28m Normal Created pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Container created 28m Normal Pulled pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Add eth0 [10.132.0.205/23] from ovn-kubernetes 28m Normal Killing pod/payload-processing-e2e-ipp-a-73f667-64d5696dc4-hddsq Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-73f667-5f44887555-s9n2j Stopping container payload-pre-processing 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Container started 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.637s (2.637s including waiting). Image size: 166612641 bytes. 28m Normal Created pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Container created 28m Normal Pulled pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.699s (2.699s including waiting). Image size: 166612641 bytes. 28m Warning Unhealthy pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Startup probe failed: Get "http://10.134.0.247:15021/healthz/ready": dial tcp 10.134.0.247:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Startup probe error: Get "http://10.134.0.247:15021/healthz/ready": dial tcp 10.134.0.247:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Readiness probe failed: Get "http://10.132.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Readiness probe error: Get "http://10.132.0.196:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-f64dc0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal Started pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Container started 28m Normal Pulled pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning Unhealthy pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 28m Normal Created pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Container created 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-f64dc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-a3619c-openshift-default Scaled up replica set e2e-shared-b-w2-a3619c-openshift-default-675f567c95 from 0 to 1 28m Normal AddedLabel gateway/e2e-shared-b-w2-a3619c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-a3619c 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-a3619c-openshift-default-675f567c95 Created pod: e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t 28m Normal AddedInterface pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Add eth0 [10.134.0.253/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-a7ca1f-openshift-default Scaled up replica set e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-a-a7ca1f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a7ca1f 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df Created pod: e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh 28m Warning Unhealthy pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Startup probe failed: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused 28m Normal AddedInterface pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Add eth0 [10.134.0.254/23] from ovn-kubernetes 28m Warning ProbeError pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Startup probe error: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused... 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-a3619c Scaled up replica set payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87 from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87 Error creating: pods "payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-a3619c: serviceaccount "payload-processing-e2e-shared-b-w2-a3619c" not found 28m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d Error creating: pods "payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-a3619c: serviceaccount "payload-processing-e2e-shared-b-w2-a3619c" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87 Created pod: payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d Created pod: payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-a3619c Scaled up replica set payload-processing-e2e-shared-b-w2-a3619c-746d6d556d from 0 to 1 28m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Container created 28m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Add eth0 [10.134.0.255/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Container started 28m Normal Pulled pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Add eth0 [10.134.1.0/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Container created 28m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Container started 28m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedLabel gateway/e2e-ipp-b-d8b2c6 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-d8b2c6 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-d8b2c6-openshift-default Scaled up replica set e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7 from 0 to 1 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7 Created pod: e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c Error creating: pods "payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a7ca1f: serviceaccount "payload-processing-e2e-ipp-a-a7ca1f" not found 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a7ca1f-8558789598 Error creating: pods "payload-processing-e2e-ipp-a-a7ca1f-8558789598-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a7ca1f: serviceaccount "payload-processing-e2e-ipp-a-a7ca1f" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a7ca1f Scaled up replica set payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c from 0 to 1 28m Normal AddedInterface pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Add eth0 [10.134.1.1/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a7ca1f Scaled up replica set payload-processing-e2e-ipp-a-a7ca1f-8558789598 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c Created pod: payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a7ca1f-8558789598 Created pod: payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h Container started 28m Normal Created pod/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj Add eth0 [10.134.1.2/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h Add eth0 [10.134.1.3/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-73435a-openshift-default-fc454477f Created pod: e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-73435a-openshift-default Scaled up replica set e2e-shared-b-w4-73435a-openshift-default-fc454477f from 1 to 2 28m Normal Started pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Container started 28m Normal AddedInterface pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Add eth0 [10.134.1.4/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Container created 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5 Created pod: payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5 Error creating: pods "payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d8b2c6: serviceaccount "payload-processing-e2e-ipp-b-d8b2c6" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b Created pod: payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b Error creating: pods "payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d8b2c6: serviceaccount "payload-processing-e2e-ipp-b-d8b2c6" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d8b2c6 Scaled up replica set payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d8b2c6 Scaled up replica set payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5 from 0 to 1 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 Container created 28m Normal Started pod/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn Container started 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn Add eth0 [10.134.1.5/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 Add eth0 [10.134.1.6/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Startup probe error: Get "http://10.134.1.4:15021/healthz/ready": dial tcp 10.134.1.4:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Startup probe failed: Get "http://10.134.1.4:15021/healthz/ready": dial tcp 10.134.1.4:15021: connect: connection refused 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-73f667 Scaled up replica set payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-73f667 Scaled up replica set payload-processing-e2e-ipp-a-73f667-9d9c68b76 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7 Created pod: payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7 Error creating: pods "payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f667: serviceaccount "payload-processing-e2e-ipp-a-73f667" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-73f667-9d9c68b76 Created pod: payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-73f667-9d9c68b76 Error creating: pods "payload-processing-e2e-ipp-a-73f667-9d9c68b76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-73f667: serviceaccount "payload-processing-e2e-ipp-a-73f667" not found 27m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-f64dc0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-f64dc0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Pulled pod/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 Add eth0 [10.134.1.7/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Add eth0 [10.133.0.173/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 Container created 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-a3619c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Container started 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-a3619c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.038s (2.038s including waiting). Image size: 166612641 bytes. 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Container created 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-73f667-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-73f667-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal AddedLabel gateway/e2e-ipp-b-6d44c2 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-6d44c2 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-6d44c2-openshift-default Scaled up replica set e2e-ipp-b-6d44c2-openshift-default-69fb7659f from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-6d44c2-openshift-default-69fb7659f Created pod: e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd 27m Normal AddedInterface pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Add eth0 [10.132.0.207/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff Error creating: pods "payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6d44c2: serviceaccount "payload-processing-e2e-ipp-b-6d44c2" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-6d44c2 Scaled up replica set payload-processing-e2e-ipp-b-6d44c2-84b45b7d48 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48 Created pod: payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48 Error creating: pods "payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6d44c2: serviceaccount "payload-processing-e2e-ipp-b-6d44c2" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-6d44c2 Scaled up replica set payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff from 0 to 1 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-beeba4-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff Created pod: payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 Add eth0 [10.134.1.8/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 Add eth0 [10.134.1.9/23] from ovn-kubernetes 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 Container started 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 Created pod: e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 Container created 27m Normal AddedInterface pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Add eth0 [10.133.0.177/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Container created 27m Normal Started pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Container started 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-beeba4-openshift-default Scaled up replica set e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 from 2 to 3 27m Normal Pulled pod/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 Container started 27m Warning ProbeError pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a5e9ab-openshift-default Scaled up replica set e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9 from 1 to 2 27m Normal Started pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Container started 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a5e9ab-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9 Created pod: e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt 27m Normal AddedInterface pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Add eth0 [10.132.0.209/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Container created 27m Warning Unhealthy pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Startup probe failed: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Startup probe error: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused... 26m Normal Started pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Container started 26m Normal Created pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Container created 26m Normal Pulled pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Pulled pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Container started 26m Normal Created pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Container created 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a7ca1f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a7ca1f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning ProbeError pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Startup probe error: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Startup probe failed: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-73435a-openshift-default Scaled up replica set e2e-shared-b-w4-73435a-openshift-default-fc454477f from 2 to 3 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-73435a-openshift-default-fc454477f Created pod: e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal AddedInterface pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Add eth0 [10.133.0.181/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Container created 26m Normal Started pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Container started 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a7ca1f-788dfbb58c-dt82h Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-a-a7ca1f-8558789598-6jrqj Stopping container payload-processing 26m Warning Unhealthy pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 26m Normal Killing pod/payload-processing-e2e-ipp-b-d8b2c6-6fd666c55b-4r756 Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bf667b7f5-rrwwn Stopping container payload-pre-processing 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-a3619c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-a3619c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a7ca1f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a7ca1f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning Unhealthy pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Readiness probe failed: Get "http://10.134.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning ProbeError pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Readiness probe error: Get "http://10.134.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Normal Created pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Container created 26m Normal Started pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Container started 26m Normal Created pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Container created 26m Normal Pulled pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Container started 26m Normal Pulled pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Startup probe failed: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Startup probe failed: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Startup probe error: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Startup probe error: Get "http://10.134.1.1:15021/healthz/ready": dial tcp 10.134.1.1:15021: connect: connection refused... 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d8b2c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d8b2c6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/payload-processing-e2e-ipp-b-6d44c2-84b45b7d48-zntz9 Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-6d44c2-9bcc874ff-j2kz4 Stopping container payload-pre-processing 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a7ca1f Scaled up replica set payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7 Error creating: pods "payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a7ca1f: serviceaccount "payload-processing-e2e-ipp-a-a7ca1f" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884 Created pod: payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884 Error creating: pods "payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a7ca1f: serviceaccount "payload-processing-e2e-ipp-a-a7ca1f" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a7ca1f Scaled up replica set payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7 Created pod: payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj 25m Normal Started pod/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj Add eth0 [10.134.1.11/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv Add eth0 [10.134.1.12/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj Container started 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-d8b2c6 Scaled up replica set payload-processing-e2e-ipp-b-d8b2c6-855bcf88db from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db Error creating: pods "payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d8b2c6: serviceaccount "payload-processing-e2e-ipp-b-d8b2c6" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-d8b2c6 Scaled up replica set payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc Error creating: pods "payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-d8b2c6: serviceaccount "payload-processing-e2e-ipp-b-d8b2c6" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc Created pod: payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db Created pod: payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 25m Normal Created pod/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp Add eth0 [10.134.1.13/23] from ovn-kubernetes 25m Normal Started pod/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 Add eth0 [10.134.1.14/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp Container started 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-01215f-openshift-default-95bfd666f Created pod: e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-01215f-openshift-default Scaled up replica set e2e-ipp-a-01215f-openshift-default-95bfd666f from 0 to 1 25m Normal AddedLabel gateway/e2e-ipp-a-01215f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-01215f 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6d44c2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal AddedInterface pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Add eth0 [10.132.0.214/23] from ovn-kubernetes 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6d44c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 25m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-4nbkc Stopping container istio-proxy 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled down replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 5 to 4 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default New size: 4; reason: All metrics below target 25m Normal Killing pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Deleted pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-q86hp 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-4nbkc 25m Normal AddedLabel gateway/e2e-ipp-b-cf4567 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-cf4567 25m Normal SuccessfulCreate replicaset/e2e-ipp-b-cf4567-openshift-default-66b97cc97f Created pod: e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d 25m Normal ScalingReplicaSet deployment/e2e-ipp-b-cf4567-openshift-default Scaled up replica set e2e-ipp-b-cf4567-openshift-default-66b97cc97f from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-01215f-9977c47d Error creating: pods "payload-processing-e2e-ipp-a-01215f-9977c47d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-01215f: serviceaccount "payload-processing-e2e-ipp-a-01215f" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-01215f-9977c47d Created pod: payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-01215f Scaled up replica set payload-processing-e2e-ipp-a-01215f-9977c47d from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-01215f Scaled up replica set payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9 Created pod: payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn 25m Normal AddedInterface pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Add eth0 [10.134.1.15/23] from ovn-kubernetes 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9 Error creating: pods "payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-01215f: serviceaccount "payload-processing-e2e-ipp-a-01215f" not found 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k Add eth0 [10.134.1.17/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn Add eth0 [10.134.1.16/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k Container started 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-cf4567-8cf8db99b Error creating: pods "payload-processing-e2e-ipp-b-cf4567-8cf8db99b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cf4567: serviceaccount "payload-processing-e2e-ipp-b-cf4567" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475 Error creating: pods "payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cf4567: serviceaccount "payload-processing-e2e-ipp-b-cf4567" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-cf4567-8cf8db99b Created pod: payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475 Created pod: payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-d8b2c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-cf4567 Scaled up replica set payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-cf4567 Scaled up replica set payload-processing-e2e-ipp-b-cf4567-8cf8db99b from 0 to 1 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-d8b2c6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd Add eth0 [10.132.0.216/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt Add eth0 [10.132.0.215/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd Container started 25m Normal Created pod/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt Container started 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default New size: 3; reason: All metrics below target 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6d44c2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6d44c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Pulled pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Container started 25m Normal Created pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Container created 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-009e78-openshift-default-5674f58c7 Created pod: e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-009e78-openshift-default Scaled up replica set e2e-shared-a-w6-009e78-openshift-default-5674f58c7 from 2 to 3 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-cce513-openshift-default Scaled down replica set e2e-shared-b-w1-cce513-openshift-default-84cc9879 from 4 to 3 25m Normal AddedInterface pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Add eth0 [10.133.0.182/23] from ovn-kubernetes 25m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-cce513-openshift-default-84cc9879 Deleted pod: e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 25m Normal Killing pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-w4gt5 Stopping container istio-proxy 25m Warning Unhealthy pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Startup probe failed: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Startup probe error: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused... 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-6d44c2 Scaled up replica set payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-6d44c2 Scaled up replica set payload-processing-e2e-ipp-b-6d44c2-76668986b6 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799 Created pod: payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-6d44c2-76668986b6 Created pod: payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-6d44c2-76668986b6 Error creating: pods "payload-processing-e2e-ipp-b-6d44c2-76668986b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6d44c2: serviceaccount "payload-processing-e2e-ipp-b-6d44c2" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799 Error creating: pods "payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6d44c2: serviceaccount "payload-processing-e2e-ipp-b-6d44c2" not found 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx Add eth0 [10.134.1.20/23] from ovn-kubernetes 25m Normal Created pod/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn Add eth0 [10.134.1.19/23] from ovn-kubernetes 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn Container started 25m Normal Pulled pod/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn Container created 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-73f667-68746cfdb7-hsxp2 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-55a4c7-85555b48dd-qlbt2 Stopping container payload-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-73f667-9d9c68b76-fr9v2 Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-55a4c7-7cd554dcbc-msrnq Stopping container payload-pre-processing 24m Normal Killing pod/e2e-ipp-a-73f667-openshift-default-7bcfd5c9b9-7fcm7 Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-55a4c7-openshift-default-759c67ccc7-7qn88 Stopping container istio-proxy 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled down replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 4 to 3 24m Normal Killing pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default New size: 3; reason: All metrics below target 24m Normal SuccessfulDelete replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Deleted pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-wjg5k 24m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-worker-w5-6f4fba-openshift-default-65f44b9749 Deleted pod: e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6f4fba-openshift-default (combined from similar events): New size: 1; reason: All metrics below target 24m Normal ScalingReplicaSet deployment/e2e-worker-w5-6f4fba-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-6f4fba-openshift-default-65f44b9749 from 2 to 1 24m Warning Unhealthy pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning ProbeError pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-dvw9n Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r Stopping container istio-proxy 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-5f387c-openshift-default New size: 1; reason: All metrics below target 24m Normal SuccessfulDelete replicaset/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59 Deleted pod: e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-bdp4r 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5f387c-openshift-default Scaled down replica set e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59 from 2 to 1 24m Normal Killing pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 Stopping container istio-proxy 24m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-cce513-openshift-default-84cc9879 Deleted pod: e2e-shared-b-w1-cce513-openshift-default-84cc9879-k66r8 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-cce513-openshift-default Scaled down replica set e2e-shared-b-w1-cce513-openshift-default-84cc9879 from 3 to 2 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-cce513-openshift-default New size: 2; reason: All metrics below target 23m Normal Killing pod/payload-processing-e2e-ipp-b-d8b2c6-855bcf88db-7qrb6 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-d8b2c6-bc6ddf8dc-j9xsp Stopping container payload-pre-processing 23m Normal AddedLabel gateway/e2e-ipp-a-f6d5b8 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-f6d5b8 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-f6d5b8-openshift-default Scaled up replica set e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77 from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77 Created pod: e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt 23m Normal AddedInterface pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Add eth0 [10.132.0.241/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5 Created pod: payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f6d5b8-865b864594 Error creating: pods "payload-processing-e2e-ipp-a-f6d5b8-865b864594-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f6d5b8: serviceaccount "payload-processing-e2e-ipp-a-f6d5b8" not found 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f6d5b8 Scaled up replica set payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f6d5b8-865b864594 Created pod: payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5 Error creating: pods "payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f6d5b8: serviceaccount "payload-processing-e2e-ipp-a-f6d5b8" not found 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f6d5b8 Scaled up replica set payload-processing-e2e-ipp-a-f6d5b8-865b864594 from 0 to 1 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 Add eth0 [10.132.0.242/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 Container started 23m Normal Killing pod/e2e-ipp-b-d8b2c6-openshift-default-749b97c7b7-vsq6g Stopping container istio-proxy 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 Add eth0 [10.132.0.243/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Warning ProbeError pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Readiness probe error: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Warning Unhealthy pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Readiness probe failed: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Normal Started pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Container started 23m Normal Created pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Container created 23m Normal Pulled pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-01215f-58f65c44f9-lf7bn Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-01215f-9977c47d-9hh6k Stopping container payload-processing 23m Normal Pulled pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Container created 23m Normal Started pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Container started 23m Normal Killing pod/payload-processing-e2e-shared-a-w0-5f387c-d78d85d96-f5csl Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-5f387c-5cf86f7b99-9gvnl Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-cf4567-8cf8db99b-rw6qt Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-cf4567-9f6f5f475-6wcvd Stopping container payload-pre-processing 23m Warning ProbeError pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Startup probe error: Get "http://10.134.1.15:15021/healthz/ready": dial tcp 10.134.1.15:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Startup probe failed: Get "http://10.134.1.15:15021/healthz/ready": dial tcp 10.134.1.15:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default New size: 2; reason: All metrics below target 23m Normal Killing pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg Stopping container istio-proxy 23m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-11f521-openshift-default Scaled down replica set e2e-shared-a-w3-11f521-openshift-default-96689b67c from 3 to 2 23m Normal Killing pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Stopping container istio-proxy 23m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-11f521-openshift-default-96689b67c Deleted pod: e2e-shared-a-w3-11f521-openshift-default-96689b67c-p9ptg 23m Warning ProbeError pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-shared-a-w0-5f387c-openshift-default-5ff776bc59-f4q9n Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-6d44c2-7bb4866799-7lhgn Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-6d44c2-76668986b6-947cx Stopping container payload-processing 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-01215f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-01215f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-00e07a-7f596f48d7-sdcgr Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-shared-b-w0-00e07a-6c7c75ff48-t2w5v Stopping container payload-processing 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-cf4567-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-cf4567-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Killing pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-b-6d44c2-openshift-default-69fb7659f-snvvd Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal Killing pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Stopping container istio-proxy 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-01215f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-01215f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Stopping container payload-pre-processing 23m Warning Unhealthy pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-shared-b-w0-00e07a-openshift-default-56d8b679cf-vf2ms Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7d9897b54c Deleted pod: maas-default-gateway-openshift-default-7d9897b54c-fv487 23m Normal Killing pod/maas-default-gateway-openshift-default-7d9897b54c-fv487 Stopping container istio-proxy 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-01215f Scaled up replica set payload-pre-processing-e2e-ipp-a-01215f-575ff4747f from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-01215f Scaled up replica set payload-processing-e2e-ipp-a-01215f-588df9f96 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f Created pod: payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-01215f-588df9f96 Error creating: pods "payload-processing-e2e-ipp-a-01215f-588df9f96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-01215f: serviceaccount "payload-processing-e2e-ipp-a-01215f" not found 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 Add eth0 [10.134.1.34/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f Error creating: pods "payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-01215f: serviceaccount "payload-processing-e2e-ipp-a-01215f" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-01215f-588df9f96 Created pod: payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w Add eth0 [10.134.1.35/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w Container started 22m Normal Created pod/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w Container created 22m Normal Killing pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Stopping container payload-processing 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-cf4567 Scaled up replica set payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-cf4567 Scaled up replica set payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8 Created pod: payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8 Error creating: pods "payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cf4567: serviceaccount "payload-processing-e2e-ipp-b-cf4567" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc Created pod: payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc Error creating: pods "payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-cf4567: serviceaccount "payload-processing-e2e-ipp-b-cf4567" not found 22m Normal Created pod/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz Add eth0 [10.134.1.37/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv Container created 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv Add eth0 [10.134.1.36/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz Container started 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv Container started 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-48df96-openshift-default New size: 2; reason: All metrics below target 22m Normal Killing pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh Stopping container istio-proxy 22m Normal SuccessfulDelete replicaset/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 Deleted pod: e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-h65wh 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-48df96-openshift-default Scaled down replica set e2e-shared-a-w1-48df96-openshift-default-5c8d99d755 from 3 to 2 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-48df96-5784ccbccf-gdg9l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-48df96-75bb88b5b4-7qhwz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-9a8001-openshift-default-d885f8f7f Created pod: e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-9a8001-openshift-default Scaled up replica set e2e-ipp-b-9a8001-openshift-default-d885f8f7f from 0 to 1 22m Normal AddedLabel gateway/e2e-ipp-b-9a8001 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-9a8001 22m Normal AddedInterface pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Add eth0 [10.132.0.249/23] from ovn-kubernetes 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-52650f-7686c45df9-r8bkm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-worker-w0-52650f-5768bb8d4b-r95hx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-5rlnk Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w1-48df96-openshift-default-5c8d99d755-8s7j6 Stopping container istio-proxy 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-9a8001 Scaled up replica set payload-processing-e2e-ipp-b-9a8001-75767f6bf6 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-9a8001 Scaled up replica set payload-pre-processing-e2e-ipp-b-9a8001-68775d7859 from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-9a8001-75767f6bf6 Error creating: pods "payload-processing-e2e-ipp-b-9a8001-75767f6bf6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9a8001: serviceaccount "payload-processing-e2e-ipp-b-9a8001" not found 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859 Error creating: pods "payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9a8001: serviceaccount "payload-processing-e2e-ipp-b-9a8001" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-9a8001-75767f6bf6 Created pod: payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859 Created pod: payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh Add eth0 [10.134.1.39/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh Container started 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 Container started 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 Add eth0 [10.134.1.38/23] from ovn-kubernetes 22m Normal Created pod/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh Container created 22m Normal Killing pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Stopping container payload-pre-processing 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-812ba2-openshift-default-779b8b57d7 Created pod: e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-812ba2-openshift-default Scaled up replica set e2e-ipp-a-812ba2-openshift-default-779b8b57d7 from 0 to 1 22m Normal AddedLabel gateway/e2e-ipp-a-812ba2 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-812ba2 22m Normal AddedInterface pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Add eth0 [10.134.1.40/23] from ovn-kubernetes 22m Normal Killing pod/payload-processing-e2e-ipp-a-f6d5b8-865b864594-d5n25 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f6d5b8-55c9ff76c5-xh6w2 Stopping container payload-pre-processing 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-812ba2 Scaled up replica set payload-processing-e2e-ipp-a-812ba2-7768d47f6d from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-812ba2 Scaled up replica set payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5 Created pod: payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5 Error creating: pods "payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-812ba2: serviceaccount "payload-processing-e2e-ipp-a-812ba2" not found 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj Add eth0 [10.132.0.253/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-812ba2-7768d47f6d Error creating: pods "payload-processing-e2e-ipp-a-812ba2-7768d47f6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-812ba2: serviceaccount "payload-processing-e2e-ipp-a-812ba2" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-812ba2-7768d47f6d Created pod: payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj 22m Normal Pulled pod/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj Container created 22m Normal Started pod/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj Container started 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 Add eth0 [10.134.1.41/23] from ovn-kubernetes 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 Container started 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-a3619c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-a3619c-openshift-default-675f567c95 Created pod: e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-a3619c-openshift-default Scaled up replica set e2e-shared-b-w2-a3619c-openshift-default-675f567c95 from 1 to 2 22m Normal Pulled pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Started pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Container started 22m Normal Created pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Container created 22m Normal AddedInterface pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Add eth0 [10.132.0.255/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Container created 22m Normal Started pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Container started 21m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-cce513-7c484dd7d8-dl5vb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-cce513-6f5d768c59-dnpdv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f6d5b8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f6d5b8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning Unhealthy pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Startup probe failed: Get "http://10.132.0.255:15021/healthz/ready": dial tcp 10.132.0.255:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Startup probe error: Get "http://10.132.0.255:15021/healthz/ready": dial tcp 10.132.0.255:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Startup probe failed: Get "http://10.132.0.241:15021/healthz/ready": dial tcp 10.132.0.241:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Startup probe error: Get "http://10.132.0.241:15021/healthz/ready": dial tcp 10.132.0.241:15021: connect: connection refused... 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-93079c-openshift-default New size: 2; reason: All metrics below target 21m Normal Killing pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp Stopping container istio-proxy 21m Normal SuccessfulDelete replicaset/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 Deleted pod: e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-x66jp 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-93079c-openshift-default Scaled down replica set e2e-shared-a-w5-93079c-openshift-default-85d87fbf79 from 3 to 2 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87 Error creating: pods "payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f6d5b8: serviceaccount "payload-processing-e2e-ipp-a-f6d5b8" not found 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87 Created pod: payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d Error creating: pods "payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-f6d5b8: serviceaccount "payload-processing-e2e-ipp-a-f6d5b8" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-f6d5b8 Scaled up replica set payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-f6d5b8 Scaled up replica set payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87 from 0 to 1 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct Add eth0 [10.134.1.43/23] from ovn-kubernetes 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 Container started 21m Normal Created pod/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct Container created 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d Created pod: payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct 21m Normal Started pod/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct Container started 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 Add eth0 [10.134.1.42/23] from ovn-kubernetes 21m Normal Pulled pod/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-b73232-openshift-default-59c5464ccc Created pod: e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 21m Normal AddedLabel gateway/e2e-ipp-a-b73232 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-b73232 21m Normal ScalingReplicaSet deployment/e2e-ipp-a-b73232-openshift-default Scaled up replica set e2e-ipp-a-b73232-openshift-default-59c5464ccc from 0 to 1 21m Normal Killing pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-ngpk9 Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-b-w1-cce513-openshift-default-84cc9879-5drrc Stopping container istio-proxy 21m Normal AddedInterface pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Add eth0 [10.132.1.1/23] from ovn-kubernetes 21m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-b7vtt Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w0-52650f-openshift-default-6f5fdf664f-8gg8x Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-b73232 Scaled up replica set payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f from 0 to 1 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-b73232-6f9854fb6 Error creating: pods "payload-processing-e2e-ipp-a-b73232-6f9854fb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b73232: serviceaccount "payload-processing-e2e-ipp-a-b73232" not found 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f Created pod: payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f Error creating: pods "payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b73232: serviceaccount "payload-processing-e2e-ipp-a-b73232" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-b73232 Scaled up replica set payload-processing-e2e-ipp-a-b73232-6f9854fb6 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-b73232-6f9854fb6 Created pod: payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg 21m Normal Started pod/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg Container started 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx Add eth0 [10.132.1.3/23] from ovn-kubernetes 21m Normal Created pod/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Pulled pod/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx Container created 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg Add eth0 [10.134.1.44/23] from ovn-kubernetes 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx Container started 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-f6d5b8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-f6d5b8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9 Created pod: e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-16c30f-openshift-default Scaled up replica set e2e-ipp-b-16c30f-openshift-default-dffb9ddc9 from 0 to 1 21m Normal AddedLabel gateway/e2e-ipp-b-16c30f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-16c30f 21m Normal AddedInterface pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Add eth0 [10.132.1.4/23] from ovn-kubernetes 21m Normal Killing pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Stopping container payload-processing 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf Created pod: payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-16c30f-55569b797b Error creating: pods "payload-processing-e2e-ipp-b-16c30f-55569b797b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-16c30f: serviceaccount "payload-processing-e2e-ipp-b-16c30f" not found 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-16c30f-55569b797b Created pod: payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf Error creating: pods "payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-16c30f: serviceaccount "payload-processing-e2e-ipp-b-16c30f" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-16c30f Scaled up replica set payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-16c30f Scaled up replica set payload-processing-e2e-ipp-b-16c30f-55569b797b from 0 to 1 21m Normal Pulled pod/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk Container started 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk Add eth0 [10.134.1.46/23] from ovn-kubernetes 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk Container created 21m Normal Started pod/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 Container started 21m Normal Created pod/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 Container created 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 Add eth0 [10.134.1.45/23] from ovn-kubernetes 21m Normal Started pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Container started 21m Normal Pulled pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Container created 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-9a8001-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-9a8001-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning ProbeError pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Startup probe error: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Startup probe failed: Get "http://10.132.0.249:15021/healthz/ready": dial tcp 10.132.0.249:15021: connect: connection refused 21m Warning Unhealthy pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/payload-processing-e2e-ipp-b-9a8001-75767f6bf6-ktwwh Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-9a8001-68775d7859-h8jm7 Stopping container payload-pre-processing 20m Normal SuccessfulCreate replicaset/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df Created pod: e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-a7ca1f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/e2e-ipp-a-a7ca1f-openshift-default Scaled up replica set e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df from 1 to 2 20m Normal Created pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Container created 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-29be73-64579fcddc-7cr2l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Started pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Container started 20m Normal AddedInterface pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Add eth0 [10.132.1.7/23] from ovn-kubernetes 20m Normal Pulled pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Warning Unhealthy pod/payload-processing-e2e-worker-w1-29be73-58c6d59956-2sszw Readiness probe failed: timeout: failed to connect service "10.133.0.63:50052" within 1s: context deadline exceeded 20m Warning ProbeError pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Startup probe error: Get "http://10.132.1.7:15021/healthz/ready": dial tcp 10.132.1.7:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Startup probe failed: Get "http://10.132.1.7:15021/healthz/ready": dial tcp 10.132.1.7:15021: connect: connection refused 20m Normal Killing pod/payload-processing-e2e-ipp-a-812ba2-7768d47f6d-bfbkj Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-812ba2-6f9b9d5fc5-dtcs5 Stopping container payload-pre-processing 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-9a8001-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-9a8001-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Pulled pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Container started 20m Normal Created pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Container created 20m Warning Unhealthy pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Startup probe failed: Get "http://10.134.1.40:15021/healthz/ready": dial tcp 10.134.1.40:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Startup probe error: Get "http://10.134.1.40:15021/healthz/ready": dial tcp 10.134.1.40:15021: connect: connection refused... 20m Normal ScalingReplicaSet deployment/e2e-worker-w1-29be73-openshift-default Scaled up replica set e2e-worker-w1-29be73-openshift-default-66467f5dcf from 2 to 4 20m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk 20m Normal SuccessfulCreate replicaset/e2e-worker-w1-29be73-openshift-default-66467f5dcf Created pod: e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-29be73-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 20m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Container started 20m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Add eth0 [10.134.1.48/23] from ovn-kubernetes 20m Normal AddedInterface pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Add eth0 [10.134.1.49/23] from ovn-kubernetes 20m Normal Pulled pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Container started 20m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Container created 20m Normal Created pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Container created 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-9a8001-646fd774b6 Error creating: pods "payload-processing-e2e-ipp-b-9a8001-646fd774b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9a8001: serviceaccount "payload-processing-e2e-ipp-b-9a8001" not found 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc Error creating: pods "payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-9a8001: serviceaccount "payload-processing-e2e-ipp-b-9a8001" not found 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-9a8001 Scaled up replica set payload-processing-e2e-ipp-b-9a8001-646fd774b6 from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-9a8001 Scaled up replica set payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc from 0 to 1 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n Container started 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc Created pod: payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c Add eth0 [10.133.0.192/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-9a8001-646fd774b6 Created pod: payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c 20m Normal Pulled pod/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n Container created 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n Add eth0 [10.133.0.191/23] from ovn-kubernetes 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-812ba2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Started pod/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c Container started 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-812ba2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Created pod/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c Container created 20m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Startup probe error: Get "http://10.134.1.49:15021/healthz/ready": dial tcp 10.134.1.49:15021: connect: connection refused... 20m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Startup probe error: Get "http://10.134.1.48:15021/healthz/ready": dial tcp 10.134.1.48:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Startup probe failed: Get "http://10.134.1.48:15021/healthz/ready": dial tcp 10.134.1.48:15021: connect: connection refused 20m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Startup probe failed: Get "http://10.134.1.49:15021/healthz/ready": dial tcp 10.134.1.49:15021: connect: connection refused 20m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-9c28b Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Stopping container istio-proxy 20m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-jwjhk Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-7qlrx Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning Unhealthy pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-worker-w1-29be73-openshift-default-66467f5dcf-5tcsp Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d Created pod: payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75 Created pod: payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-812ba2 Scaled up replica set payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75 from 0 to 1 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d Error creating: pods "payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-812ba2: serviceaccount "payload-processing-e2e-ipp-a-812ba2" not found 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75 Error creating: pods "payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-812ba2: serviceaccount "payload-processing-e2e-ipp-a-812ba2" not found 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-812ba2 Scaled up replica set payload-processing-e2e-ipp-a-812ba2-596b9f7c9d from 0 to 1 20m Normal Started pod/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp Container started 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 Container started 20m Normal Created pod/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp Container created 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 Container created 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp Add eth0 [10.134.1.50/23] from ovn-kubernetes 20m Normal Pulled pod/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 Add eth0 [10.133.0.193/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Deleted pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-7f92f6-openshift-default Scaled down replica set e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 from 5 to 4 20m Normal Killing pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-plcp4 Stopping container istio-proxy 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-7f92f6-openshift-default New size: 4; reason: All metrics below target 20m Warning ProbeError pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Readiness probe error: Get "http://10.132.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 20m Warning Unhealthy pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Readiness probe failed: Get "http://10.132.1.1:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 20m Normal Created pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Container created 20m Normal Pulled pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Pulled pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Container started 20m Normal Created pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Container created 20m Normal Started pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Container started 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-b73232-7f6ff6589f-nmvjx Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-b73232-6f9854fb6-l79cg Stopping container payload-processing 19m Warning ProbeError pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Startup probe error: Get "http://10.132.1.4:15021/healthz/ready": dial tcp 10.132.1.4:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Startup probe failed: Get "http://10.132.1.4:15021/healthz/ready": dial tcp 10.132.1.4:15021: connect: connection refused 19m Warning Unhealthy pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Startup probe failed: Get "http://10.132.1.1:15021/healthz/ready": dial tcp 10.132.1.1:15021: connect: connection refused 19m Warning ProbeError pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Startup probe error: Get "http://10.132.1.1:15021/healthz/ready": dial tcp 10.132.1.1:15021: connect: connection refused... 19m Normal AddedLabel gateway/e2e-ipp-b-1e8c0e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-1e8c0e 19m Normal SuccessfulCreate replicaset/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f Created pod: e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q 19m Normal ScalingReplicaSet deployment/e2e-ipp-b-1e8c0e-openshift-default Scaled up replica set e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f from 0 to 1 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-812ba2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-812ba2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal AddedInterface pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Add eth0 [10.132.1.11/23] from ovn-kubernetes 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-b73232-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-b73232-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-16c30f-f89465fcf-zc9dk Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-16c30f-55569b797b-xbxw5 Stopping container payload-processing 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7 Error creating: pods "payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1e8c0e: serviceaccount "payload-processing-e2e-ipp-b-1e8c0e" not found 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678 Error creating: pods "payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1e8c0e: serviceaccount "payload-processing-e2e-ipp-b-1e8c0e" not found 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678 Created pod: payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1e8c0e Scaled up replica set payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7 from 0 to 1 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1e8c0e Scaled up replica set payload-processing-e2e-ipp-b-1e8c0e-857cdc4678 from 0 to 1 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7 Created pod: payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 19m Normal Started pod/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 Container started 19m Normal Pulled pod/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Created pod/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 Container created 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 Add eth0 [10.134.1.51/23] from ovn-kubernetes 19m Normal Started pod/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw Container started 19m Normal Created pod/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw Container created 19m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw Add eth0 [10.132.1.12/23] from ovn-kubernetes 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-16c30f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-16c30f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a7ca1f-7dd98fb4d7-28zsj Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-a7ca1f-66cb4d7884-6k2gv Stopping container payload-processing 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-16c30f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-16c30f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-01215f-575ff4747f-xcmr5 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-01215f-588df9f96-9rc5w Stopping container payload-processing 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-b73232-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-b73232-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-b73232 Scaled up replica set payload-pre-processing-e2e-ipp-a-b73232-6848d99b47 from 0 to 1 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-b73232 Scaled up replica set payload-processing-e2e-ipp-a-b73232-f54cbdc7b from 0 to 1 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-b73232-f54cbdc7b Error creating: pods "payload-processing-e2e-ipp-a-b73232-f54cbdc7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b73232: serviceaccount "payload-processing-e2e-ipp-a-b73232" not found 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh Add eth0 [10.134.1.52/23] from ovn-kubernetes 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47 Created pod: payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47 Error creating: pods "payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-b73232: serviceaccount "payload-processing-e2e-ipp-a-b73232" not found 19m Normal AddedInterface pod/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr Add eth0 [10.134.1.53/23] from ovn-kubernetes 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-b73232-f54cbdc7b Created pod: payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr 19m Normal Created pod/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh Container created 19m Normal Started pod/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh Container started 19m Normal Pulled pod/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Normal Created pod/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr Container created 19m Normal Started pod/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr Container started 19m Normal Killing pod/e2e-ipp-a-01215f-openshift-default-95bfd666f-rkm78 Stopping container istio-proxy 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-beeba4-openshift-default New size: 2; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-beeba4-openshift-default Scaled down replica set e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 from 3 to 2 19m Normal Killing pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt Stopping container istio-proxy 19m Normal SuccessfulDelete replicaset/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957 Deleted pod: e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-x49mt 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-16c30f Scaled up replica set payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c from 0 to 1 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4 Error creating: pods "payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-16c30f: serviceaccount "payload-processing-e2e-ipp-b-16c30f" not found 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c Error creating: pods "payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-16c30f: serviceaccount "payload-processing-e2e-ipp-b-16c30f" not found 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-16c30f Scaled up replica set payload-processing-e2e-ipp-b-16c30f-78c96f8ff4 from 0 to 1 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs Add eth0 [10.134.1.54/23] from ovn-kubernetes 19m Normal Pulled pod/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4 Created pod: payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx 19m Normal Created pod/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs Container created 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c Created pod: payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs 19m Normal AddedInterface pod/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx Add eth0 [10.133.0.194/23] from ovn-kubernetes 19m Normal Started pod/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs Container started 19m Normal Started pod/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx Container started 19m Normal Created pod/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx Container created 19m Normal AddedLabel gateway/e2e-ipp-b-658c99 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-658c99 19m Normal SuccessfulCreate replicaset/e2e-ipp-b-658c99-openshift-default-54d78694df Created pod: e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj 19m Normal ScalingReplicaSet deployment/e2e-ipp-b-658c99-openshift-default Scaled up replica set e2e-ipp-b-658c99-openshift-default-54d78694df from 0 to 1 18m Normal AddedInterface pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Add eth0 [10.132.1.14/23] from ovn-kubernetes 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9 Error creating: pods "payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-658c99: serviceaccount "payload-processing-e2e-ipp-b-658c99" not found 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-658c99 Scaled up replica set payload-processing-e2e-ipp-b-658c99-7bdc65f946 from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-658c99 Scaled up replica set payload-pre-processing-e2e-ipp-b-658c99-77d45498f9 from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9 Created pod: payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-658c99-7bdc65f946 Error creating: pods "payload-processing-e2e-ipp-b-658c99-7bdc65f946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-658c99: serviceaccount "payload-processing-e2e-ipp-b-658c99" not found 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-658c99-7bdc65f946 Created pod: payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch 18m Normal Created pod/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch Container started 18m Normal Pulled pod/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch Add eth0 [10.134.1.56/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr Container started 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr Add eth0 [10.134.1.55/23] from ovn-kubernetes 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr Container created 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-9a8001-6f5b8674dc-xch8n Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-b-9a8001-646fd774b6-5xz5c Stopping container payload-processing 18m Normal Killing pod/e2e-ipp-b-9a8001-openshift-default-d885f8f7f-dt8r5 Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-t4shh Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-ipp-a-a7ca1f-openshift-default-77b7bf58df-l8qlx Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-93079c-7d66db76d7-ls977 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-a-w5-93079c-668d8c594d-qtx2p Stopping container payload-processing 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-658c99-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-658c99-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Started pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Container started 18m Normal Created pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Container created 18m Normal Pulled pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Warning Unhealthy pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Startup probe failed: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1e8c0e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning ProbeError pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Startup probe error: Get "http://10.132.1.11:15021/healthz/ready": dial tcp 10.132.1.11:15021: connect: connection refused... 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1e8c0e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Killing pod/payload-processing-e2e-ipp-b-1e8c0e-857cdc4678-hxkmw Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1e8c0e-6dcdd584d7-q8vw2 Stopping container payload-pre-processing 18m Normal Killing pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-7km72 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w5-93079c-openshift-default-85d87fbf79-8gwxc Stopping container istio-proxy 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-f6d5b8-86876b7f87-7gzr9 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-f6d5b8-76dcb85c6d-77gct Stopping container payload-processing 17m Normal Killing pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Stopping container istio-proxy 17m Warning ProbeError pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Stopping container payload-pre-processing 17m Warning Unhealthy pod/e2e-ipp-a-f6d5b8-openshift-default-68ddb87d77-n4kjt Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal Killing pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Stopping container payload-processing 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1e8c0e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1e8c0e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-cf4567-57b7896bdc-d9hwv Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-cf4567-5dbcbb6f8-lwkvz Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-16c30f-7b54948c4c-2cpqs Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-16c30f-78c96f8ff4-dwhmx Stopping container payload-processing 17m Normal Started pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Container started 17m Normal Created pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Container created 17m Normal Pulled pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1e8c0e Scaled up replica set payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff from 0 to 1 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1e8c0e Scaled up replica set payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d from 0 to 1 17m Normal Killing pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Stopping container istio-proxy 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-11f521-openshift-default New size: 1; reason: All metrics below target 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff Created pod: payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk 17m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-11f521-openshift-default Scaled down replica set e2e-shared-a-w3-11f521-openshift-default-96689b67c from 2 to 1 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d Error creating: pods "payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1e8c0e: serviceaccount "payload-processing-e2e-ipp-b-1e8c0e" not found 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d Created pod: payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk Add eth0 [10.134.1.58/23] from ovn-kubernetes 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-11f521-openshift-default-96689b67c Deleted pod: e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 17m Normal Killing pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-74286 Stopping container istio-proxy 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff Error creating: pods "payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1e8c0e: serviceaccount "payload-processing-e2e-ipp-b-1e8c0e" not found 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm Add eth0 [10.134.1.59/23] from ovn-kubernetes 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk Container created 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk Container started 17m Normal Pulled pod/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Created pod/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm Container created 17m Normal Started pod/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm Container started 17m Warning Unhealthy pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Startup probe failed: Get "http://10.132.1.14:15021/healthz/ready": dial tcp 10.132.1.14:15021: connect: connection refused 17m Warning ProbeError pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Startup probe error: Get "http://10.132.1.14:15021/healthz/ready": dial tcp 10.132.1.14:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-ipp-b-cf4567-openshift-default-66b97cc97f-bhb7d Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz Stopping container istio-proxy 17m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-73435a-openshift-default Scaled down replica set e2e-shared-b-w4-73435a-openshift-default-fc454477f from 3 to 2 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-73435a-openshift-default New size: 2; reason: All metrics below target 17m Normal Killing pod/e2e-ipp-b-16c30f-openshift-default-dffb9ddc9-6jkdj Stopping container istio-proxy 17m Normal SuccessfulDelete replicaset/e2e-shared-b-w4-73435a-openshift-default-fc454477f Deleted pod: e2e-shared-b-w4-73435a-openshift-default-fc454477f-5kjsz 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-a5e9ab-74df65b974-xdkp9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-a5e9ab-5d6f79c8c9-5rg9c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-11f521-6f8968b75-h6zh8 Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-a-w3-11f521-7d8bfbf78-q7qwq Stopping container payload-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-658c99-7bdc65f946-t89ch Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-658c99-77d45498f9-ftcpr Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Stopping container payload-pre-processing 17m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-009e78-openshift-default Scaled down replica set e2e-shared-a-w6-009e78-openshift-default-5674f58c7 from 3 to 2 17m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-009e78-openshift-default-5674f58c7 Deleted pod: e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp 17m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-009e78-openshift-default New size: 2; reason: All metrics below target 17m Normal Killing pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-znnhp Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-m58bh Stopping container istio-proxy 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-658c99-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-658c99-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning ProbeError pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-shared-b-w5-a5e9ab-openshift-default-8498d5b8b9-vr4jt Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-beeba4-c8c5c5566-rqsqh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-beeba4-7d8695bc7f-xqctm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-pre-processing-e2e-worker-w5-6f4fba-5c479c7dd7-9sxbh Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-worker-w5-6f4fba-86877b49f8-n59lf Stopping container payload-processing 17m Normal Killing pod/e2e-shared-a-w3-11f521-openshift-default-96689b67c-7jp2h Stopping container istio-proxy 16m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-658c99 Scaled up replica set payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55 from 0 to 1 16m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-658c99 Scaled up replica set payload-processing-e2e-ipp-b-658c99-8578c5b85b from 0 to 1 16m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-658c99-8578c5b85b Error creating: pods "payload-processing-e2e-ipp-b-658c99-8578c5b85b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-658c99: serviceaccount "payload-processing-e2e-ipp-b-658c99" not found 16m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55 Error creating: pods "payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-658c99: serviceaccount "payload-processing-e2e-ipp-b-658c99" not found 16m Normal AddedInterface pod/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt Add eth0 [10.134.1.65/23] from ovn-kubernetes 16m Normal Pulled pod/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-658c99-8578c5b85b Created pod: payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt 16m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg Add eth0 [10.134.1.64/23] from ovn-kubernetes 16m Normal Created pod/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt Container created 16m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55 Created pod: payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg 16m Normal Started pod/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt Container started 16m Normal Started pod/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg Container started 16m Normal Created pod/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg Container created 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Stopping container payload-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-812ba2-596b9f7c9d-gtlmp Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-812ba2-5f95998c75-v5lq9 Stopping container payload-pre-processing 16m Normal Killing pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-hmlbk Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-a-w4-beeba4-openshift-default-6ff99b6957-xkhd8 Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Stopping container istio-proxy 16m Warning Unhealthy pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-ipp-a-812ba2-openshift-default-779b8b57d7-v75n2 Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Stopping container payload-processing 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-8e0f66-fc8567cdc-knftm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-8e0f66-8646cb7ccd-fw5mb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-processing-e2e-ipp-a-b73232-f54cbdc7b-5mlgr Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-b73232-6848d99b47-4xmfh Stopping container payload-pre-processing 16m Normal Killing pod/e2e-shared-b-w3-8e0f66-openshift-default-75fdc5ccf7-c7q4q Stopping container istio-proxy 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-73435a-766b94d975-d8ztv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-73435a-7645c9bc64-wcm9t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-processing-e2e-ipp-b-1e8c0e-6b88cf5b8d-txnvm Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1e8c0e-649b5b8dff-k47pk Stopping container payload-pre-processing 16m Normal Killing pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-2mhcb Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-a-b73232-openshift-default-59c5464ccc-2kpc7 Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w4-73435a-openshift-default-fc454477f-x6g2b Stopping container istio-proxy 16m Normal Killing pod/e2e-worker-w5-6f4fba-openshift-default-65f44b9749-cbs5z Stopping container istio-proxy 16m Normal Killing pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Stopping container istio-proxy 16m Normal Killing pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Stopping container payload-pre-processing 16m Warning Unhealthy pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-ipp-b-1e8c0e-openshift-default-6fb676f64f-ld62q Readiness probe error: HTTP probe failed with statuscode: 503... 16m Normal Killing pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Stopping container payload-pre-processing 16m Normal Killing pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v Stopping container istio-proxy 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w2-a3619c-openshift-default-675f567c95 Deleted pod: e2e-shared-b-w2-a3619c-openshift-default-675f567c95-jsx2v 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-a3619c-openshift-default New size: 1; reason: All metrics below target 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-a3619c-openshift-default Scaled down replica set e2e-shared-b-w2-a3619c-openshift-default-675f567c95 from 2 to 1 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-7f92f6-openshift-default Scaled down replica set e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 from 4 to 2 16m Normal Killing pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 Stopping container istio-proxy 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-7f92f6-openshift-default New size: 2; reason: All metrics below target 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Deleted pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-nqdp4 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969 Deleted pod: e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh 16m Normal Killing pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-d28kh Stopping container istio-proxy 15m Normal Killing pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Stopping container payload-pre-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-658c99-5dd6d56c55-gn8lg Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-ipp-b-658c99-8578c5b85b-z8slt Stopping container payload-processing 15m Warning Unhealthy pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-ipp-b-658c99-openshift-default-54d78694df-cfhwj Stopping container istio-proxy 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-f64dc0-b4f9c4899-cnjhz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-f64dc0-8555d77c9d-sxwmp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-bbb7b Stopping container istio-proxy 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-cefb27-dfdc89d75-jm8hc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Stopping container istio-proxy 15m Warning Unhealthy pod/payload-processing-e2e-worker-w3-cefb27-88db76d58-8gzgh Readiness probe failed: timeout: failed to connect service "10.133.0.57:50052" within 1s: context deadline exceeded 15m Warning Unhealthy pod/payload-processing-e2e-worker-w4-ca5363-657fbd45cb-wc7w2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-ca5363-57d48d9d99-ctxwr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-worker-w3-cefb27-openshift-default-577b649d97-6m5cj Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/e2e-shared-a-w2-f64dc0-openshift-default-59d7879cd9-f5nrs Stopping container istio-proxy 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Stopping container payload-processing 15m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-fjbg5 Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Stopping container istio-proxy 15m Warning ProbeError pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/e2e-worker-w4-ca5363-openshift-default-6cfd85d975-xfr4r Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Readiness probe error: HTTP probe failed with statuscode: 503... 15m Normal Killing pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-9fkcq Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-a-w6-009e78-openshift-default-5674f58c7-dgw5w Readiness probe failed: HTTP probe failed with statuscode: 503 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-7f92f6-79df4f4c6c-c4qwp Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-shared-b-w6-7f92f6-5744474458-tvxhl Stopping container payload-processing 15m Normal Killing pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Stopping container istio-proxy 15m Warning Unhealthy pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Readiness probe failed: HTTP probe failed with statuscode: 503 15m Warning ProbeError pod/e2e-shared-b-w2-a3619c-openshift-default-675f567c95-7ts8t Readiness probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-009e78-847cbfbf7d-vgf6m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-009e78-54d78f6cf9-sjgcl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-a3619c-746d6d556d-rxcbw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-a3619c-6584cd5f87-cgfd2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-m9zn4 Stopping container istio-proxy 15m Normal Killing pod/e2e-shared-b-w6-7f92f6-openshift-default-85bfc49969-rnd66 Stopping container istio-proxy 15m Normal Killing pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Stopping container payload-pre-processing 15m Normal Killing pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Stopping container payload-processing 14m Normal Killing pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Stopping container payload-pre-processing 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-1a11cf-b489f44f9-lwz7b Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-worker-w2-1a11cf-6d77fdc4b4-tqjbm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-zfzfv Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Stopping container istio-proxy 14m Warning ProbeError pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/e2e-worker-w2-1a11cf-openshift-default-748ddd859-947lv Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-021d6d-67c98cf9c9-qkfq7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-worker-w6-021d6d-765d889d96-p2p24 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-mclhg Stopping container istio-proxy 14m Normal Killing pod/e2e-worker-w6-021d6d-openshift-default-65c58b6947-lrlct Stopping container istio-proxy 14m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-srsm4 14m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default (combined from similar events): New size: 4; reason: cpu resource utilization (percentage of request) above target 14m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-kgxxw 13m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Container created 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Add eth0 [10.134.1.85/23] from ovn-kubernetes 13m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Container started 13m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Container created 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Add eth0 [10.134.1.84/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Container started 13m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Startup probe failed: Get "http://10.134.1.85:15021/healthz/ready": dial tcp 10.134.1.85:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Startup probe error: Get "http://10.134.1.85:15021/healthz/ready": dial tcp 10.134.1.85:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Startup probe failed: Get "http://10.134.1.84:15021/healthz/ready": dial tcp 10.134.1.84:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Startup probe error: Get "http://10.134.1.84:15021/healthz/ready": dial tcp 10.134.1.84:15021: connect: connection refused... 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default (combined from similar events): Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 4 to 6 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-2jzrh 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-26zqk 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Add eth0 [10.133.0.199/23] from ovn-kubernetes 13m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Container started 13m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Container created 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Add eth0 [10.132.1.25/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Container created 13m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Container started 13m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Startup probe failed: Get "http://10.132.1.25:15021/healthz/ready": dial tcp 10.132.1.25:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Startup probe failed: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Startup probe error: Get "http://10.132.1.25:15021/healthz/ready": dial tcp 10.132.1.25:15021: connect: connection refused... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Startup probe error: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused... 12m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-lvppb 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-ccb2j 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-97m56 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 12m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Add eth0 [10.134.1.87/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Container started 12m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Add eth0 [10.132.1.26/23] from ovn-kubernetes 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Add eth0 [10.133.0.200/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Container created 12m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Container created 12m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Container created 12m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Container started 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Add eth0 [10.134.1.88/23] from ovn-kubernetes 12m Normal Started pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Container started 12m Normal Created pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Container created 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7d9897b54c Created pod: maas-default-gateway-openshift-default-7d9897b54c-hb9g9 12m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7d9897b54c from 6 to 10 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Startup probe failed: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Startup probe failed: Get "http://10.134.1.87:15021/healthz/ready": dial tcp 10.134.1.87:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Startup probe error: Get "http://10.134.1.88:15021/healthz/ready": dial tcp 10.134.1.88:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Startup probe error: Get "http://10.132.1.26:15021/healthz/ready": dial tcp 10.132.1.26:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Startup probe error: Get "http://10.134.1.87:15021/healthz/ready": dial tcp 10.134.1.87:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Startup probe failed: Get "http://10.134.1.88:15021/healthz/ready": dial tcp 10.134.1.88:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Startup probe failed: Get "http://10.132.1.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Startup probe error: Get "http://10.132.1.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Startup probe failed: Get "http://10.134.1.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Startup probe error: Get "http://10.134.1.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Startup probe error: Get "http://10.134.1.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Startup probe failed: Get "http://10.134.1.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-k5nr4 12m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-k5nr4 Stopping container discovery 12m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 12m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 9m20s Normal AddedInterface pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Add eth0 [10.133.0.201/23] from ovn-kubernetes 9m20s Normal AddedLabel gateway/e2e-worker-single-main-3ab7f1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-3ab7f1 9m20s Normal SuccessfulCreate replicaset/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd9 Created pod: e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn 9m20s Normal ScalingReplicaSet deployment/e2e-worker-single-main-3ab7f1-openshift-default Scaled up replica set e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd9 from 0 to 1 9m15s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d87bd7 Created pod: payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd 9m15s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96748 Created pod: payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 9m15s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96748 Error creating: pods "payload-processing-e2e-worker-single-main-3ab7f1-5f47d96748-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-3ab7f1: serviceaccount "payload-processing-e2e-worker-single-main-3ab7f1" not found 9m15s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d87bd7 Error creating: pods "payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d87bd7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-3ab7f1: serviceaccount "payload-processing-e2e-worker-single-main-3ab7f1" not found 9m15s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-3ab7f1 Scaled up replica set payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d87bd7 from 0 to 1 9m15s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-3ab7f1 Scaled up replica set payload-processing-e2e-worker-single-main-3ab7f1-5f47d96748 from 0 to 1 9m15s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Add eth0 [10.133.0.203/23] from ovn-kubernetes 9m14s Normal Created pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Container created 9m14s Normal Started pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Container started 9m14s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 9m14s Normal Created pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Container created 9m14s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Add eth0 [10.133.0.204/23] from ovn-kubernetes 9m14s Normal Pulled pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 9m14s Normal Started pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Container started 9m3s Warning ProbeError pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Readiness probe error: Get "http://10.133.0.201:15021/healthz/ready": read tcp 10.133.0.2:56142->10.133.0.201:15021: read: connection reset by peer... 9m3s Warning Unhealthy pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Readiness probe failed: Get "http://10.133.0.201:15021/healthz/ready": read tcp 10.133.0.2:56142->10.133.0.201:15021: read: connection reset by peer 9m2s Normal Started pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Container started 9m2s Normal Created pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Container created 9m2s Normal Pulled pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m57s Warning Unhealthy pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 8m57s Warning ProbeError pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 8m50s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-3ab7f1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m50s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-3ab7f1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 8m20s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-3ab7f1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m20s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-3ab7f1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m15s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Stopping container payload-pre-processing 8m15s Normal Killing pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Stopping container payload-processing 8m5s Normal Killing pod/e2e-worker-single-main-3ab7f1-openshift-default-7884d75fd99dlrn Stopping container istio-proxy 7m52s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-3ab7f1-f54d8jfdkd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m52s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-3ab7f1-5f47d96747ls65 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m10s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Readiness probe failed: Get "http://10.132.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m10s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-pdgh4 Readiness probe error: Get "http://10.132.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m40s Normal SuccessfulCreate replicaset/e2e-worker-single-main-fe8356-openshift-default-586db97f7 Created pod: e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk 5m40s Normal AddedLabel gateway/e2e-worker-single-main-fe8356 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-fe8356 5m40s Normal ScalingReplicaSet deployment/e2e-worker-single-main-fe8356-openshift-default Scaled up replica set e2e-worker-single-main-fe8356-openshift-default-586db97f7 from 0 to 1 5m39s Normal AddedInterface pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Add eth0 [10.134.1.106/23] from ovn-kubernetes 5m35s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-fe8356 Scaled up replica set payload-processing-e2e-worker-single-main-fe8356-79cd4c6974 from 0 to 1 5m34s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bb8459 Created pod: payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx 5m34s Normal Pulled pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m34s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Add eth0 [10.133.0.209/23] from ovn-kubernetes 5m34s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m34s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-fe8356-79cd4c6974 Created pod: payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 5m34s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-fe8356-79cd4c6974 Error creating: pods "payload-processing-e2e-worker-single-main-fe8356-79cd4c6974-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-fe8356: serviceaccount "payload-processing-e2e-worker-single-main-fe8356" not found 5m34s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bb8459 Error creating: pods "payload-pre-processing-e2e-worker-single-main-fe8356-57d4bb8459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-fe8356: serviceaccount "payload-processing-e2e-worker-single-main-fe8356" not found 5m34s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Add eth0 [10.133.0.208/23] from ovn-kubernetes 5m34s Normal Created pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Container created 5m34s Normal Started pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Container started 5m34s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-fe8356 Scaled up replica set payload-pre-processing-e2e-worker-single-main-fe8356-57d4bb8459 from 0 to 1 5m33s Normal Created pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Container created 5m33s Normal Started pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Container started 5m17s Normal Created pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Container created 5m17s Normal Started pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Container started 5m17s Normal Pulled pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m10s Warning ProbeError pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Startup probe error: Get "http://10.134.1.106:15021/healthz/ready": dial tcp 10.134.1.106:15021: connect: connection refused... 5m10s Warning Unhealthy pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Startup probe failed: Get "http://10.134.1.106:15021/healthz/ready": dial tcp 10.134.1.106:15021: connect: connection refused 5m10s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-fe8356-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m10s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-fe8356-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m40s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-fe8356-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m40s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-fe8356-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m38s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Stopping container payload-pre-processing 4m38s Normal Killing pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Stopping container payload-processing 4m28s Normal Killing pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Stopping container istio-proxy 4m24s Warning ProbeError pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Readiness probe error: HTTP probe failed with statuscode: 503... 4m24s Warning Unhealthy pod/e2e-worker-single-main-fe8356-openshift-default-586db97f7-zm8jk Readiness probe failed: HTTP probe failed with statuscode: 503 4m22s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-fe8356-57d4bpvtqx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m22s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-fe8356-79cd4c697vj9c4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m2s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Readiness probe error: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 4m2s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Readiness probe error: Get "http://10.132.1.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 4m2s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Readiness probe failed: Get "http://10.134.1.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 4m2s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-ccb2j Readiness probe error: Get "http://10.134.1.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 4m2s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-97m56 Readiness probe failed: Get "http://10.133.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 4m2s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-hb9g9 Readiness probe failed: Get "http://10.132.1.26:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 4m2s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Readiness probe error: Get "http://10.134.1.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 4m2s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-lvppb Readiness probe failed: Get "http://10.134.1.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m27s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Readiness probe error: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m27s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-cngvb Readiness probe failed: Get "http://10.133.0.40:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 110s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Readiness probe error: Get "http://10.132.1.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 110s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-2jzrh Readiness probe failed: Get "http://10.132.1.25:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 109s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Readiness probe failed: Get "http://10.133.0.199:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 109s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Readiness probe failed: Get "http://10.134.1.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 109s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-srsm4 Readiness probe error: Get "http://10.134.1.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 109s Warning Unhealthy pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Readiness probe failed: Get "http://10.134.1.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 109s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-26zqk Readiness probe error: Get "http://10.133.0.199:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 109s Warning ProbeError pod/maas-default-gateway-openshift-default-7d9897b54c-kgxxw Readiness probe error: Get "http://10.134.1.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...