=== events in openshift-ingress (2026-10-30T03:46:52Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 47m Normal Scheduled pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Successfully assigned openshift-ingress/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r to ip-10-0-136-171.ec2.internal 31m Normal Scheduled pod/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 to ip-10-0-136-238.ec2.internal 46m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm to ip-10-0-128-214.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs to ip-10-0-136-171.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl to ip-10-0-136-171.ec2.internal 41m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 to ip-10-0-136-171.ec2.internal 22m Normal Scheduled pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Successfully assigned openshift-ingress/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 to ip-10-0-136-238.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p to ip-10-0-136-171.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v to ip-10-0-136-238.ec2.internal 51m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f to ip-10-0-128-214.ec2.internal 47m Normal Scheduled pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Successfully assigned openshift-ingress/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt to ip-10-0-128-214.ec2.internal 26m Normal Scheduled pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Successfully assigned openshift-ingress/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk Successfully assigned openshift-ingress/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk to ip-10-0-128-214.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-ait-adca906b-565c86788b-klv42 Successfully assigned openshift-ingress/payload-processing-e2e-ait-adca906b-565c86788b-klv42 to ip-10-0-128-214.ec2.internal 44m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p to ip-10-0-136-238.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 to ip-10-0-128-214.ec2.internal 54m Normal Scheduled pod/payload-processing-5f78687d5b-v8fgz Successfully assigned openshift-ingress/payload-processing-5f78687d5b-v8fgz to ip-10-0-136-238.ec2.internal 51m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j to ip-10-0-136-171.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc to ip-10-0-136-171.ec2.internal 46m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 to ip-10-0-136-238.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 to ip-10-0-128-214.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Successfully assigned openshift-ingress/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Successfully assigned openshift-ingress/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp to ip-10-0-136-238.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg to ip-10-0-128-214.ec2.internal 47m Normal Scheduled pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Successfully assigned openshift-ingress/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z to ip-10-0-136-238.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 to ip-10-0-136-238.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m to ip-10-0-136-171.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s to ip-10-0-136-171.ec2.internal 52m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl to ip-10-0-136-238.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf to ip-10-0-136-171.ec2.internal 51m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m to ip-10-0-136-171.ec2.internal 46m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln to ip-10-0-136-171.ec2.internal 48m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 to ip-10-0-136-238.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf to ip-10-0-128-214.ec2.internal 44m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 to ip-10-0-136-171.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Successfully assigned openshift-ingress/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 to ip-10-0-136-171.ec2.internal 38m Normal Scheduled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Successfully assigned openshift-ingress/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 to ip-10-0-136-238.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx to ip-10-0-128-214.ec2.internal 45m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp to ip-10-0-136-171.ec2.internal 43m Normal Scheduled pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Successfully assigned openshift-ingress/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d to ip-10-0-136-238.ec2.internal 45m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp to ip-10-0-136-238.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl to ip-10-0-136-238.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q to ip-10-0-136-171.ec2.internal 46m Normal Scheduled pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Successfully assigned openshift-ingress/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b to ip-10-0-136-171.ec2.internal 39m Normal Scheduled pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Successfully assigned openshift-ingress/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b to ip-10-0-136-171.ec2.internal 51m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z to ip-10-0-128-214.ec2.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s to ip-10-0-136-171.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl to ip-10-0-136-238.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 to ip-10-0-128-214.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff to ip-10-0-128-214.ec2.internal 39m Normal Scheduled pod/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d Successfully assigned openshift-ingress/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d to ip-10-0-136-171.ec2.internal 24m Normal Scheduled pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Successfully assigned openshift-ingress/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r to ip-10-0-128-214.ec2.internal 35m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b to ip-10-0-136-238.ec2.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 to ip-10-0-128-214.ec2.internal 45m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 to ip-10-0-128-214.ec2.internal 41m Warning FailedScheduling pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr Successfully assigned openshift-ingress/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr to ip-10-0-136-171.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n to ip-10-0-136-171.ec2.internal 41m Normal Scheduled pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Successfully assigned openshift-ingress/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 to ip-10-0-136-171.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv to ip-10-0-136-171.ec2.internal 26m Normal Scheduled pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Successfully assigned openshift-ingress/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx to ip-10-0-136-171.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj to ip-10-0-136-238.ec2.internal 34m Normal Scheduled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Successfully assigned openshift-ingress/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t to ip-10-0-136-171.ec2.internal 48m Normal Scheduled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Successfully assigned openshift-ingress/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx to ip-10-0-128-214.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 to ip-10-0-128-214.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 to ip-10-0-136-171.ec2.internal 39m Normal Scheduled pod/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j to ip-10-0-136-171.ec2.internal 18m Normal Scheduled pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Successfully assigned openshift-ingress/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g to ip-10-0-136-238.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp Successfully assigned openshift-ingress/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp to ip-10-0-136-171.ec2.internal 41m Normal Scheduled pod/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j Successfully assigned openshift-ingress/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j to ip-10-0-136-171.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp to ip-10-0-136-238.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 to ip-10-0-128-214.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd to ip-10-0-136-238.ec2.internal 43m Normal Scheduled pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Successfully assigned openshift-ingress/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 to ip-10-0-128-214.ec2.internal 25m Normal Scheduled pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Successfully assigned openshift-ingress/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs to ip-10-0-136-171.ec2.internal 42m Warning FailedScheduling pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb Successfully assigned openshift-ingress/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb to ip-10-0-136-171.ec2.internal 50m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr to ip-10-0-136-171.ec2.internal 8m55s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs to ip-10-0-136-238.ec2.internal 5m25s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz to ip-10-0-136-238.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh to ip-10-0-136-171.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f to ip-10-0-136-171.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv to ip-10-0-136-238.ec2.internal 60m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 to ip-10-0-128-214.ec2.internal 20m Normal Scheduled pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Successfully assigned openshift-ingress/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht to ip-10-0-128-214.ec2.internal 54m Normal Scheduled pod/payload-pre-processing-7b98bb678c-8tfqx Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-8tfqx to ip-10-0-136-171.ec2.internal 25m Normal Scheduled pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Successfully assigned openshift-ingress/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh to ip-10-0-136-238.ec2.internal 40m Normal Scheduled pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Successfully assigned openshift-ingress/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 Successfully assigned openshift-ingress/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 to ip-10-0-128-214.ec2.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 to ip-10-0-128-214.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 to ip-10-0-136-238.ec2.internal 44m Normal Scheduled pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Successfully assigned openshift-ingress/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg to ip-10-0-136-238.ec2.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr to ip-10-0-136-171.ec2.internal 20m Normal Scheduled pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Successfully assigned openshift-ingress/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl to ip-10-0-136-238.ec2.internal 22m Normal Scheduled pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Successfully assigned openshift-ingress/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 to ip-10-0-136-171.ec2.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-v4xpn Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-v4xpn to ip-10-0-136-171.ec2.internal 32m Normal Scheduled pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx to ip-10-0-136-171.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv to ip-10-0-128-214.ec2.internal 37m Normal Scheduled pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 to ip-10-0-136-171.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl to ip-10-0-136-171.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 to ip-10-0-136-171.ec2.internal 37m Normal Scheduled pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Successfully assigned openshift-ingress/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b to ip-10-0-136-171.ec2.internal 49m Normal Scheduled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Successfully assigned openshift-ingress/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv to ip-10-0-136-171.ec2.internal 19m Normal Scheduled pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Successfully assigned openshift-ingress/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg to ip-10-0-136-171.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt to ip-10-0-136-238.ec2.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs to ip-10-0-136-238.ec2.internal 24m Normal Scheduled pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Successfully assigned openshift-ingress/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 to ip-10-0-128-214.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 to ip-10-0-136-171.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 to ip-10-0-136-238.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 to ip-10-0-136-238.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld to ip-10-0-136-238.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn to ip-10-0-136-171.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 to ip-10-0-136-238.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 to ip-10-0-136-238.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 to ip-10-0-136-171.ec2.internal 41m Normal Scheduled pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Successfully assigned openshift-ingress/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf to ip-10-0-136-171.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Successfully assigned openshift-ingress/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt to ip-10-0-136-171.ec2.internal 21m Normal Scheduled pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Successfully assigned openshift-ingress/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb to ip-10-0-128-214.ec2.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp to ip-10-0-136-238.ec2.internal 26m Normal Scheduled pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Successfully assigned openshift-ingress/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h to ip-10-0-136-171.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-c22dw to ip-10-0-128-214.ec2.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-zprng to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-mhnwk to ip-10-0-128-214.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 to ip-10-0-136-171.ec2.internal 19m Normal Scheduled pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Successfully assigned openshift-ingress/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf to ip-10-0-136-238.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 to ip-10-0-136-171.ec2.internal 51m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Successfully assigned openshift-ingress/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl to ip-10-0-136-238.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 to ip-10-0-136-171.ec2.internal 34m Normal Scheduled pod/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k to ip-10-0-136-171.ec2.internal 52m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c to ip-10-0-128-214.ec2.internal 33m Normal Scheduled pod/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 to ip-10-0-128-214.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 to ip-10-0-136-171.ec2.internal 27m Normal Scheduled pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Successfully assigned openshift-ingress/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb to ip-10-0-136-238.ec2.internal 59m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-t7cnx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-t7cnx to ip-10-0-136-171.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-qv642 to ip-10-0-136-171.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn to ip-10-0-136-171.ec2.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf to ip-10-0-136-238.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 to ip-10-0-136-171.ec2.internal 43m Normal Scheduled pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Successfully assigned openshift-ingress/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr to ip-10-0-128-214.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb to ip-10-0-136-171.ec2.internal 51m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc to ip-10-0-136-171.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Successfully assigned openshift-ingress/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 to ip-10-0-136-238.ec2.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc to ip-10-0-136-238.ec2.internal 48m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-5fnmn to ip-10-0-136-238.ec2.internal 47m Normal Scheduled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Successfully assigned openshift-ingress/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m to ip-10-0-128-214.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 to ip-10-0-136-238.ec2.internal 49m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w to ip-10-0-128-214.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z to ip-10-0-128-214.ec2.internal 28m Normal Scheduled pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Successfully assigned openshift-ingress/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 to ip-10-0-136-171.ec2.internal 49m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j to ip-10-0-136-171.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Successfully assigned openshift-ingress/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh to ip-10-0-136-171.ec2.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d to ip-10-0-136-171.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s to ip-10-0-136-171.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c to ip-10-0-136-171.ec2.internal 48m Normal Scheduled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Successfully assigned openshift-ingress/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 to ip-10-0-136-171.ec2.internal 38m Normal Scheduled pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Successfully assigned openshift-ingress/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r to ip-10-0-136-171.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj to ip-10-0-128-214.ec2.internal 43m Normal Scheduled pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Successfully assigned openshift-ingress/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx to ip-10-0-136-171.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf to ip-10-0-128-214.ec2.internal 12m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f to ip-10-0-128-214.ec2.internal 44m Normal Scheduled pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Successfully assigned openshift-ingress/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb to ip-10-0-128-214.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs to ip-10-0-128-214.ec2.internal 25m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf to ip-10-0-136-238.ec2.internal 41m Normal Scheduled pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Successfully assigned openshift-ingress/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn to ip-10-0-136-171.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd to ip-10-0-128-214.ec2.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd to ip-10-0-136-238.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 to ip-10-0-136-238.ec2.internal 17m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 to ip-10-0-136-238.ec2.internal 43m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 to ip-10-0-136-171.ec2.internal 60m Normal Scheduled pod/kube-auth-proxy-64f884f994-j22s7 Successfully assigned openshift-ingress/kube-auth-proxy-64f884f994-j22s7 to ip-10-0-136-171.ec2.internal 32m Normal Scheduled pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Successfully assigned openshift-ingress/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz to ip-10-0-136-171.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Successfully assigned openshift-ingress/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn to ip-10-0-136-238.ec2.internal 28m Normal Scheduled pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Successfully assigned openshift-ingress/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 to ip-10-0-128-214.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Successfully assigned openshift-ingress/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 to ip-10-0-136-238.ec2.internal 51m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q to ip-10-0-136-238.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz to ip-10-0-128-214.ec2.internal 48m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 to ip-10-0-136-238.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz to ip-10-0-136-238.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm to ip-10-0-136-238.ec2.internal 66m Normal Scheduled pod/router-default-6b959b9795-jw4rz Successfully assigned openshift-ingress/router-default-6b959b9795-jw4rz to ip-10-0-136-238.ec2.internal 66m Warning FailedScheduling pod/router-default-6b959b9795-jw4rz no nodes available to schedule pods 74m Warning FailedScheduling pod/router-default-6b959b9795-jw4rz no nodes available to schedule pods 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm to ip-10-0-136-171.ec2.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr to ip-10-0-136-171.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv to ip-10-0-136-238.ec2.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 to ip-10-0-136-171.ec2.internal 33m Normal Scheduled pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Successfully assigned openshift-ingress/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf to ip-10-0-136-238.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 to ip-10-0-136-238.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn to ip-10-0-136-171.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 to ip-10-0-136-238.ec2.internal 60m Normal Scheduled pod/kube-auth-proxy-64f884f994-jv42m Successfully assigned openshift-ingress/kube-auth-proxy-64f884f994-jv42m to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl to ip-10-0-136-171.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 to ip-10-0-136-238.ec2.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 to ip-10-0-136-238.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn to ip-10-0-136-171.ec2.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v to ip-10-0-136-171.ec2.internal 37m Normal Scheduled pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Successfully assigned openshift-ingress/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm to ip-10-0-136-171.ec2.internal 54m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn to ip-10-0-136-238.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j to ip-10-0-136-238.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md to ip-10-0-136-171.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k to ip-10-0-136-171.ec2.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb to ip-10-0-136-171.ec2.internal 9m1s Normal Scheduled pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Successfully assigned openshift-ingress/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l to ip-10-0-136-171.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Successfully assigned openshift-ingress/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf to ip-10-0-136-171.ec2.internal 29m Normal Scheduled pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Successfully assigned openshift-ingress/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 to ip-10-0-128-214.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 to ip-10-0-136-238.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 to ip-10-0-136-171.ec2.internal 17m Normal Scheduled pod/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk to ip-10-0-136-238.ec2.internal 18m Normal Scheduled pod/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf to ip-10-0-128-214.ec2.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 to ip-10-0-136-238.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-734274-84746f954-z25tg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-734274-84746f954-z25tg to ip-10-0-136-238.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh to ip-10-0-136-238.ec2.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w to ip-10-0-136-171.ec2.internal 51m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm to ip-10-0-128-214.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj Successfully assigned openshift-ingress/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx to ip-10-0-136-171.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Successfully assigned openshift-ingress/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l to ip-10-0-136-238.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt to ip-10-0-128-214.ec2.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns to ip-10-0-136-171.ec2.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl to ip-10-0-136-171.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk to ip-10-0-136-238.ec2.internal 45m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn to ip-10-0-136-238.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 to ip-10-0-136-238.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv to ip-10-0-136-238.ec2.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 to ip-10-0-136-171.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Successfully assigned openshift-ingress/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 to ip-10-0-128-214.ec2.internal 43m Normal Scheduled pod/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 Successfully assigned openshift-ingress/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 to ip-10-0-136-238.ec2.internal 32m Normal Scheduled pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Successfully assigned openshift-ingress/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Successfully assigned openshift-ingress/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp to ip-10-0-136-238.ec2.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw to ip-10-0-128-214.ec2.internal 18m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf to ip-10-0-136-238.ec2.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 to ip-10-0-128-214.ec2.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 to ip-10-0-136-238.ec2.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 to ip-10-0-136-238.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 to ip-10-0-136-238.ec2.internal 52m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 to ip-10-0-136-238.ec2.internal 47m Normal Scheduled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Successfully assigned openshift-ingress/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz to ip-10-0-136-171.ec2.internal 5m30s Normal Scheduled pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Successfully assigned openshift-ingress/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 to ip-10-0-136-171.ec2.internal 36m Normal Scheduled pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Successfully assigned openshift-ingress/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m to ip-10-0-136-171.ec2.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb to ip-10-0-136-171.ec2.internal 36m Normal Scheduled pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Successfully assigned openshift-ingress/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 to ip-10-0-136-238.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 to ip-10-0-136-171.ec2.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp to ip-10-0-136-238.ec2.internal 40m Normal Scheduled pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr Successfully assigned openshift-ingress/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr to ip-10-0-136-171.ec2.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb to ip-10-0-136-171.ec2.internal 32m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c to ip-10-0-128-214.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn to ip-10-0-136-238.ec2.internal 45m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-54ldl to ip-10-0-136-238.ec2.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg to ip-10-0-136-171.ec2.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v to ip-10-0-136-171.ec2.internal 13m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f to ip-10-0-128-214.ec2.internal 50m Normal Scheduled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Successfully assigned openshift-ingress/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 to ip-10-0-136-171.ec2.internal 24m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w to ip-10-0-136-171.ec2.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw to ip-10-0-128-214.ec2.internal 35m Normal Scheduled pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Successfully assigned openshift-ingress/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg to ip-10-0-136-171.ec2.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm to ip-10-0-136-171.ec2.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s to ip-10-0-136-238.ec2.internal 26m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs to ip-10-0-136-238.ec2.internal 28m Normal Scheduled pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Successfully assigned openshift-ingress/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 to ip-10-0-128-214.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 to ip-10-0-136-171.ec2.internal 30m Normal Scheduled pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Successfully assigned openshift-ingress/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg to ip-10-0-128-214.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c to ip-10-0-136-171.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h to ip-10-0-136-238.ec2.internal 31m Normal Scheduled pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Successfully assigned openshift-ingress/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m to ip-10-0-136-171.ec2.internal 35m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks to ip-10-0-136-171.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn to ip-10-0-136-238.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc to ip-10-0-136-171.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv to ip-10-0-136-171.ec2.internal 28m Normal Scheduled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Successfully assigned openshift-ingress/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn to ip-10-0-128-214.ec2.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw to ip-10-0-136-238.ec2.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 to ip-10-0-136-238.ec2.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 to ip-10-0-128-214.ec2.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx to ip-10-0-136-238.ec2.internal 50m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p to ip-10-0-136-171.ec2.internal 28m Normal Scheduled pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Successfully assigned openshift-ingress/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh to ip-10-0-136-238.ec2.internal 44m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d to ip-10-0-136-171.ec2.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh to ip-10-0-136-238.ec2.internal 49m Normal Scheduled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Successfully assigned openshift-ingress/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd to ip-10-0-128-214.ec2.internal 34m Normal Scheduled pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Successfully assigned openshift-ingress/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw to ip-10-0-136-171.ec2.internal 53m Normal Scheduled pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 to ip-10-0-128-214.ec2.internal 8m55s Normal Scheduled pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 to ip-10-0-136-171.ec2.internal 5m25s Normal Scheduled pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c to ip-10-0-136-171.ec2.internal 22m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 to ip-10-0-136-171.ec2.internal 22m Normal Scheduled pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Successfully assigned openshift-ingress/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 to ip-10-0-136-171.ec2.internal 23m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx to ip-10-0-136-171.ec2.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb to ip-10-0-136-171.ec2.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn to ip-10-0-136-171.ec2.internal 33m Normal Scheduled pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Successfully assigned openshift-ingress/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk to ip-10-0-136-171.ec2.internal 26m Normal Scheduled pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Successfully assigned openshift-ingress/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b to ip-10-0-136-238.ec2.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h to ip-10-0-136-171.ec2.internal 24m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 to ip-10-0-136-171.ec2.internal 74m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-6b959b9795 from 0 to 1 74m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 74m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 74m Normal SuccessfulCreate replicaset/router-default-6b959b9795 Created pod: router-default-6b959b9795-jw4rz 74m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 66m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 66m Warning FailedMount pod/router-default-6b959b9795-jw4rz MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-6b959b9795-jw4rz MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 65m Warning FailedMount pod/router-default-6b959b9795-jw4rz MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 65m Warning FailedMount pod/router-default-6b959b9795-jw4rz MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 65m Normal AddedInterface pod/router-default-6b959b9795-jw4rz Add eth0 [10.132.0.8/23] from ovn-kubernetes 65m Normal Pulled pod/router-default-6b959b9795-jw4rz Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 65m Normal Started pod/router-default-6b959b9795-jw4rz Container started 65m Normal Created pod/router-default-6b959b9795-jw4rz Container created 60m Normal Pulling pod/kube-auth-proxy-64f884f994-j22s7 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal AddedInterface pod/kube-auth-proxy-64f884f994-j22s7 Add eth0 [10.133.0.23/23] from ovn-kubernetes 60m Normal Pulling pod/kube-auth-proxy-64f884f994-jv42m Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal AddedInterface pod/kube-auth-proxy-64f884f994-jv42m Add eth0 [10.132.0.31/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-64f884f994 Created pod: kube-auth-proxy-64f884f994-j22s7 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-64f884f994 Created pod: kube-auth-proxy-64f884f994-jv42m 60m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-64f884f994 from 0 to 2 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-v4xpn 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 60m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-v4xpn Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 60m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-v4xpn Add eth0 [10.133.0.24/23] from ovn-kubernetes 60m Normal Started pod/kube-auth-proxy-64f884f994-jv42m Container started 60m Normal Created pod/kube-auth-proxy-64f884f994-j22s7 Container created 60m Normal Started pod/kube-auth-proxy-64f884f994-j22s7 Container started 60m Normal Pulled pod/kube-auth-proxy-64f884f994-jv42m Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.5s (2.5s including waiting). Image size: 188904677 bytes. 60m Normal Created pod/kube-auth-proxy-64f884f994-jv42m Container created 60m Normal Pulled pod/kube-auth-proxy-64f884f994-j22s7 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.466s (2.466s including waiting). Image size: 188904677 bytes. 60m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-v4xpn Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.33s (2.33s including waiting). Image size: 185476477 bytes. 60m Normal Started pod/istiod-openshift-gateway-7876bfcb74-v4xpn Container started 60m Normal Created pod/istiod-openshift-gateway-7876bfcb74-v4xpn Container created 60m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-98f7774f Created pod: data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 60m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-98f7774f from 0 to 1 60m Normal Pulling pod/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 60m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 Add eth0 [10.134.0.25/23] from ovn-kubernetes 60m Normal Pulled pod/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.859s (2.859s including waiting). Image size: 183555333 bytes. 60m Normal Started pod/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 Container started 60m Normal Created pod/data-science-gateway-data-science-gateway-class-98f7774f-tkhp2 Container created 60m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 59m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-t7cnx 59m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-t7cnx Add eth0 [10.133.0.26/23] from ovn-kubernetes 59m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-t7cnx Container started 59m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-t7cnx Container created 59m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-t7cnx Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 59m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-v4xpn Stopping container discovery 59m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-v4xpn 59m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 0 to 1 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-w7qxw 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Add eth0 [10.134.0.33/23] from ovn-kubernetes 55m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Container started 55m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Container created 55m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused... 55m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 54m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 54m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-8tfqx 54m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-5f78687d5b from 0 to 1 54m Warning FailedCreate replicaset/payload-processing-5f78687d5b Error creating: pods "payload-processing-5f78687d5b-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 54m Normal SuccessfulCreate replicaset/payload-processing-5f78687d5b Created pod: payload-processing-5f78687d5b-v8fgz 54m Normal Pulling pod/payload-processing-5f78687d5b-v8fgz Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 54m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-8tfqx Add eth0 [10.133.0.48/23] from ovn-kubernetes 54m Normal AddedInterface pod/payload-processing-5f78687d5b-v8fgz Add eth0 [10.132.0.44/23] from ovn-kubernetes 54m Normal Pulling pod/payload-pre-processing-7b98bb678c-8tfqx Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 54m Normal Pulled pod/payload-processing-5f78687d5b-v8fgz Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.999s (1.999s including waiting). Image size: 125317101 bytes. 54m Normal Created pod/payload-pre-processing-7b98bb678c-8tfqx Container created 54m Normal Started pod/payload-pre-processing-7b98bb678c-8tfqx Container started 54m Normal Started pod/payload-processing-5f78687d5b-v8fgz Container started 54m Normal Created pod/payload-processing-5f78687d5b-v8fgz Container created 54m Normal Pulled pod/payload-pre-processing-7b98bb678c-8tfqx Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.947s (1.947s including waiting). Image size: 125317101 bytes. 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-hc2qp 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 1 to 4 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-v5g8v 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-6qcgn 54m Normal Pulling pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Add eth0 [10.132.0.45/23] from ovn-kubernetes 54m Normal Pulling pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal Pulling pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Add eth0 [10.132.0.46/23] from ovn-kubernetes 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Add eth0 [10.133.0.49/23] from ovn-kubernetes 54m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Container created 54m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.93s (2.93s including waiting). Image size: 183555333 bytes. 54m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Container created 54m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.81s (2.81s including waiting). Image size: 183555333 bytes. 54m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Container started 54m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Container created 54m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 2.402s (2.402s including waiting). Image size: 183555333 bytes. 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Startup probe error: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused... 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Startup probe error: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Startup probe failed: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Startup probe error: Get "http://10.132.0.45:15021/healthz/ready": dial tcp 10.132.0.45:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Startup probe failed: Get "http://10.132.0.46:15021/healthz/ready": dial tcp 10.132.0.46:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Startup probe failed: Get "http://10.133.0.49:15021/healthz/ready": dial tcp 10.133.0.49:15021: connect: connection refused 54m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: 54m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 6 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Add eth0 [10.133.0.51/23] from ovn-kubernetes 54m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Container created 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 54m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Container created 54m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-fp9dv 54m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Container started 54m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Container started 54m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Add eth0 [10.134.0.36/23] from ovn-kubernetes 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Startup probe error: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused... 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Startup probe failed: Get "http://10.133.0.51:15021/healthz/ready": dial tcp 10.133.0.51:15021: connect: connection refused 54m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused 54m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused... 53m Normal AddedLabel gateway/e2e-worker-w5-f7d1a1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-f7d1a1 53m Normal AddedLabel gateway/e2e-worker-w6-32bad3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-32bad3 53m Normal AddedLabel gateway/e2e-worker-w3-d9f3ac Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-d9f3ac 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled up replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled up replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 0 to 1 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled up replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 0 to 1 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 53m Normal AddedLabel gateway/e2e-worker-w1-e11f03 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-e11f03 53m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Add eth0 [10.132.0.47/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl 53m Normal AddedLabel gateway/e2e-worker-w0-aeff1d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-aeff1d 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled up replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 0 to 1 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled up replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 0 to 1 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled up replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 0 to 1 53m Normal AddedLabel gateway/e2e-worker-w4-7d1788 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-7d1788 53m Normal AddedLabel gateway/e2e-worker-w2-810119 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-810119 53m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Add eth0 [10.132.0.48/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Add eth0 [10.133.0.52/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Add eth0 [10.133.0.54/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Add eth0 [10.132.0.50/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Add eth0 [10.132.0.49/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q 53m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Add eth0 [10.133.0.53/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled up replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-32bad3 Scaled up replica set payload-processing-e2e-worker-w6-32bad3-56784c8f76 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-32bad3-56784c8f76 Created pod: payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995 Error creating: pods "payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-32bad3: serviceaccount "payload-processing-e2e-worker-w6-32bad3" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995 Created pod: payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-32bad3-56784c8f76 Error creating: pods "payload-processing-e2e-worker-w6-32bad3-56784c8f76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-32bad3: serviceaccount "payload-processing-e2e-worker-w6-32bad3" not found 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-32bad3 Scaled up replica set payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995 from 0 to 1 53m Normal Started pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Container started 53m Normal Started pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Container started 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Add eth0 [10.133.0.55/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Container created 53m Normal Created pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Container created 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Add eth0 [10.133.0.56/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-d9f3ac Scaled up replica set payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9 Error creating: pods "payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-d9f3ac: serviceaccount "payload-processing-e2e-worker-w3-d9f3ac" not found 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6 Created pod: payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6 Error creating: pods "payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-d9f3ac: serviceaccount "payload-processing-e2e-worker-w3-d9f3ac" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-d9f3ac Scaled up replica set payload-processing-e2e-worker-w3-d9f3ac-745464ddc6 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9 Created pod: payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Container started 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Add eth0 [10.133.0.57/23] from ovn-kubernetes 53m Normal Created pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Container created 53m Normal Created pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Container created 53m Normal AddedInterface pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Add eth0 [10.132.0.52/23] from ovn-kubernetes 53m Normal Pulled pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Container started 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-f7d1a1 Scaled up replica set payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b Created pod: payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f Created pod: payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b Error creating: pods "payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-f7d1a1: serviceaccount "payload-processing-e2e-worker-w5-f7d1a1" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-f7d1a1 Scaled up replica set payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f Error creating: pods "payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-f7d1a1: serviceaccount "payload-processing-e2e-worker-w5-f7d1a1" not found 53m Normal Pulled pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Container created 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Add eth0 [10.133.0.59/23] from ovn-kubernetes 53m Normal Created pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Container created 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Started pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Container started 53m Normal AddedInterface pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Add eth0 [10.133.0.58/23] from ovn-kubernetes 53m Normal Started pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Container started 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5 Created pod: payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd Created pod: payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd Error creating: pods "payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-810119: serviceaccount "payload-processing-e2e-worker-w2-810119" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-810119 Scaled up replica set payload-processing-e2e-worker-w2-810119-7fc6f9f6cd from 0 to 1 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-810119 Scaled up replica set payload-pre-processing-e2e-worker-w2-810119-7985cbbff5 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5 Error creating: pods "payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-810119: serviceaccount "payload-processing-e2e-worker-w2-810119" not found 53m Normal Created pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Container created 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5 Error creating: pods "payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-e11f03: serviceaccount "payload-processing-e2e-worker-w1-e11f03" not found 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b Error creating: pods "payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-e11f03: serviceaccount "payload-processing-e2e-worker-w1-e11f03" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5 Created pod: payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t 53m Normal Started pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Container started 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-e11f03 Scaled up replica set payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5 from 0 to 1 53m Normal Pulled pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Add eth0 [10.132.0.54/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Add eth0 [10.133.0.60/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b Created pod: payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-e11f03 Scaled up replica set payload-processing-e2e-worker-w1-e11f03-855bbc8d7b from 0 to 1 53m Normal Started pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Container started 53m Normal Created pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Container created 53m Normal Started pod/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc Container started 53m Normal Pulled pod/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Add eth0 [10.133.0.61/23] from ovn-kubernetes 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Container created 53m Normal Started pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Container started 53m Normal Created pod/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc Container created 53m Normal AddedInterface pod/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc Add eth0 [10.133.0.62/23] from ovn-kubernetes 53m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-aeff1d Scaled up replica set payload-processing-e2e-worker-w0-aeff1d-77d496d48d from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-aeff1d-77d496d48d Created pod: payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-aeff1d-77d496d48d Error creating: pods "payload-processing-e2e-worker-w0-aeff1d-77d496d48d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-aeff1d: serviceaccount "payload-processing-e2e-worker-w0-aeff1d" not found 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7 Created pod: payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-aeff1d Scaled up replica set payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7 from 0 to 1 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7 Error creating: pods "payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-aeff1d: serviceaccount "payload-processing-e2e-worker-w0-aeff1d" not found 53m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 53m Normal Pulling pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 53m Normal Started pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Container started 53m Normal Created pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Container created 53m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-7d1788-784cfd57f6 Created pod: payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx 53m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-7d1788 Scaled up replica set payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55 from 0 to 1 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Add eth0 [10.133.0.63/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Add eth0 [10.134.0.37/23] from ovn-kubernetes 53m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-7d1788-784cfd57f6 Error creating: pods "payload-processing-e2e-worker-w4-7d1788-784cfd57f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-7d1788: serviceaccount "payload-processing-e2e-worker-w4-7d1788" not found 53m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-7d1788 Scaled up replica set payload-processing-e2e-worker-w4-7d1788-784cfd57f6 from 0 to 1 53m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55 Created pod: payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55 Error creating: pods "payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-7d1788: serviceaccount "payload-processing-e2e-worker-w4-7d1788" not found 53m Normal Pulling pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 53m Normal Started pod/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx Container started 53m Normal Pulled pod/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 53m Normal Created pod/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx Container created 53m Normal AddedInterface pod/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx Add eth0 [10.133.0.64/23] from ovn-kubernetes 53m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Add eth0 [10.134.0.38/23] from ovn-kubernetes 53m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Readiness probe error: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 53m Normal Started pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Container started 53m Normal Created pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Container created 53m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Readiness probe failed: Get "http://10.132.0.47:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 53m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.05s (1.05s including waiting). Image size: 125317101 bytes. 53m Normal Pulled pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.074s (2.074s including waiting). Image size: 125317101 bytes. 53m Normal Created pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Container created 53m Normal Started pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Container started 53m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Container started 53m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Container created 53m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Startup probe failed: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Startup probe error: Get "http://10.132.0.47:15021/healthz/ready": dial tcp 10.132.0.47:15021: connect: connection refused... 53m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Container started 53m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Container created 53m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Container created 53m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Container started 53m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Container created 53m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Container started 53m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Startup probe error: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Startup probe failed: Get "http://10.133.0.53:15021/healthz/ready": dial tcp 10.133.0.53:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Startup probe error: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Startup probe failed: Get "http://10.133.0.52:15021/healthz/ready": dial tcp 10.133.0.52:15021: connect: connection refused 53m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Container started 53m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Startup probe error: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Startup probe failed: Get "http://10.133.0.54:15021/healthz/ready": dial tcp 10.133.0.54:15021: connect: connection refused 53m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Container created 53m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Container created 53m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Container started 52m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Startup probe error: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Startup probe failed: Get "http://10.132.0.48:15021/healthz/ready": dial tcp 10.132.0.48:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Startup probe error: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Startup probe failed: Get "http://10.132.0.49:15021/healthz/ready": dial tcp 10.132.0.49:15021: connect: connection refused 52m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Container created 52m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Container started 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Startup probe failed: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Startup probe error: Get "http://10.132.0.50:15021/healthz/ready": dial tcp 10.132.0.50:15021: connect: connection refused... 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Add eth0 [10.133.0.67/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled up replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 1 to 2 52m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Container created 52m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Container started 52m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled up replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 1 to 2 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled up replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 1 to 2 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Container created 52m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Add eth0 [10.134.0.46/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Container started 52m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Container created 52m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Add eth0 [10.132.0.61/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Container started 52m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Startup probe error: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Startup probe failed: Get "http://10.132.0.61:15021/healthz/ready": dial tcp 10.132.0.61:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Startup probe error: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Startup probe failed: Get "http://10.134.0.46:15021/healthz/ready": dial tcp 10.134.0.46:15021: connect: connection refused 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled up replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 1 to 2 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p 51m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Add eth0 [10.133.0.70/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Container created 51m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Container started 51m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled up replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 1 to 2 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Add eth0 [10.132.0.63/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Container created 51m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Container started 51m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Startup probe error: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Startup probe failed: Get "http://10.132.0.63:15021/healthz/ready": dial tcp 10.132.0.63:15021: connect: connection refused 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled up replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 1 to 3 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled up replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 1 to 2 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Container created 51m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Container created 51m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Container started 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled up replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 2 to 4 51m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Add eth0 [10.134.0.53/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm 51m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Container started 51m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Add eth0 [10.133.0.75/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Add eth0 [10.134.0.54/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Container started 51m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Container created 51m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Add eth0 [10.133.0.76/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Container created 51m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Container started 51m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Container created 51m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Container started 51m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Add eth0 [10.133.0.77/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Startup probe failed: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Startup probe error: Get "http://10.134.0.54:15021/healthz/ready": dial tcp 10.134.0.54:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled up replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 2 to 3 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 51m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Add eth0 [10.134.0.57/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Add eth0 [10.132.0.64/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled up replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 2 to 4 51m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Container created 51m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Container started 51m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Container created 51m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Container started 51m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Add eth0 [10.133.0.80/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Container created 51m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Container started 51m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Startup probe error: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Startup probe failed: Get "http://10.134.0.57:15021/healthz/ready": dial tcp 10.134.0.57:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Startup probe failed: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Startup probe error: Get "http://10.132.0.64:15021/healthz/ready": dial tcp 10.132.0.64:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled up replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 2 to 4 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw 50m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Container created 50m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Container started 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled up replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 2 to 3 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Add eth0 [10.134.0.59/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Container started 50m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Add eth0 [10.134.0.60/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Container started 50m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Add eth0 [10.133.0.82/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Container created 50m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Container created 50m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled up replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 3 to 4 50m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled up replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 4 to 7 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd 50m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Add eth0 [10.133.0.84/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Container created 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 50m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Container started 50m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Add eth0 [10.134.0.61/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Container started 50m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Add eth0 [10.132.0.68/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Container created 50m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Add eth0 [10.133.0.85/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Container created 50m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Container started 50m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Container created 50m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Container started 50m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Startup probe error: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Startup probe failed: Get "http://10.132.0.68:15021/healthz/ready": dial tcp 10.132.0.68:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b 50m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled up replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 3 to 4 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Add eth0 [10.133.0.86/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Container created 50m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Container started 50m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb 50m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled up replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 4 to 8 50m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 50m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Container created 50m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Container created 50m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Add eth0 [10.134.0.63/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Container created 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z 50m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp 50m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Add eth0 [10.132.0.72/23] from ovn-kubernetes 50m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Add eth0 [10.133.0.88/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled up replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 4 to 8 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 50m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Add eth0 [10.132.0.70/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Container created 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 50m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Add eth0 [10.133.0.87/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Container created 50m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Container started 50m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Container started 50m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Add eth0 [10.132.0.71/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Container started 50m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl 50m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Container started 50m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Add eth0 [10.134.0.64/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Add eth0 [10.132.0.73/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled up replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 7 to 9 50m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Container started 50m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Container started 50m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Container created 50m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Container started 50m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Container created 50m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Add eth0 [10.132.0.74/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Container created 50m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Container started 50m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Container created 50m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Container started 50m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Add eth0 [10.134.0.65/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Container created 50m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Container started 50m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 50m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Startup probe failed: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Startup probe error: Get "http://10.132.0.74:15021/healthz/ready": dial tcp 10.132.0.74:15021: connect: connection refused... 49m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Add eth0 [10.132.0.75/23] from ovn-kubernetes 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 49m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Add eth0 [10.134.0.68/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled up replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 4 to 6 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd 49m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Container created 49m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Container started 49m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Container created 49m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Container started 49m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Add eth0 [10.134.0.67/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Container created 49m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Container started 49m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled up replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 8 to 9 49m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 9; reason: All metrics below target 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Readiness probe failed: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Readiness probe error: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 49m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled up replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 8 to 10 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f 49m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Add eth0 [10.134.0.71/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f (combined from similar events): Created pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv 49m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Add eth0 [10.134.0.70/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Container created 49m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Add eth0 [10.133.0.97/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Container created 49m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Container created 49m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Container started 49m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Container started 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Add eth0 [10.133.0.96/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled up replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 9 to 10 49m Normal AddedInterface pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Add eth0 [10.132.0.76/23] from ovn-kubernetes 49m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Container started 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn 49m Normal SuccessfulCreate replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f (combined from similar events): Created pod: e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j 49m Normal Started pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Container started 49m Normal Created pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Container created 49m Normal Pulled pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled up replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 6 to 10 49m Normal AddedInterface pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Add eth0 [10.133.0.95/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz 49m Normal SuccessfulCreate replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 (combined from similar events): Created pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md 49m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Container started 49m Normal Started pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Container started 49m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Container created 49m Normal Created pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Container created 49m Normal AddedInterface pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Add eth0 [10.133.0.98/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Container created 49m Normal Started pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Container started 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 49m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused 49m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": dial tcp 10.133.0.98:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Startup probe failed: Get "http://10.133.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Startup probe error: Get "http://10.133.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-w7qxw Stopping container istio-proxy 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-hc2qp 48m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-w7qxw 48m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 6 to 4 48m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-hc2qp Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled up replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 9 to 10 48m Normal SuccessfulCreate replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 (combined from similar events): Created pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 48m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled up replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 2 to 4 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 48m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx 48m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Container started 48m Normal AddedInterface pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Add eth0 [10.133.0.103/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Add eth0 [10.132.0.77/23] from ovn-kubernetes 48m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Container created 48m Normal Pulled pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Add eth0 [10.134.0.75/23] from ovn-kubernetes 48m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Container created 48m Normal Started pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Container started 48m Normal Created pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Container created 48m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Container started 48m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Startup probe failed: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused 48m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Startup probe failed: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused 48m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Startup probe error: Get "http://10.132.0.77:15021/healthz/ready": dial tcp 10.132.0.77:15021: connect: connection refused... 48m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Startup probe error: Get "http://10.133.0.103:15021/healthz/ready": dial tcp 10.133.0.103:15021: connect: connection refused... 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled up replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 4 to 5 48m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 48m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Add eth0 [10.132.0.80/23] from ovn-kubernetes 48m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Container started 48m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Container created 48m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Readiness probe failed: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Readiness probe error: Get "http://10.134.0.63:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 48m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-5fnmn 48m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 48m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 48m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Add eth0 [10.132.0.81/23] from ovn-kubernetes 48m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 48m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Readiness probe error: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Readiness probe failed: Get "http://10.133.0.52:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Readiness probe error: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Readiness probe failed: Get "http://10.133.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Container started 47m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 8.161s (8.161s including waiting). Image size: 185476477 bytes. 47m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Container created 47m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Readiness probe failed: HTTP probe failed with statuscode: 503 47m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 47m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Stopping container istio-proxy 47m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Add eth0 [10.133.0.116/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled up replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 3 to 4 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw 47m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Container created 47m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Add eth0 [10.133.0.117/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff (combined from similar events): Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 47m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Container started 47m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled up replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 5 to 10 47m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Container created 47m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Add eth0 [10.134.0.84/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Add eth0 [10.133.0.118/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal SuccessfulCreate replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Created pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb 47m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Container started 47m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Container created 47m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Add eth0 [10.134.0.83/23] from ovn-kubernetes 47m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Container started 47m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Container created 47m Normal Pulled pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Add eth0 [10.132.0.82/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Container started 47m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Container created 47m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Container started 47m Normal Created pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Container created 47m Normal Started pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Container started 47m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Readiness probe error: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Readiness probe failed: Get "http://10.132.0.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Readiness probe error: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-q5gc5 Readiness probe failed: Get "http://10.133.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Readiness probe failed: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Readiness probe error: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Readiness probe error: Get "http://10.134.0.46:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Readiness probe failed: Get "http://10.132.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Readiness probe error: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Readiness probe failed: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Readiness probe failed: Get "http://10.133.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Readiness probe error: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Readiness probe error: Get "http://10.132.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Readiness probe failed: Get "http://10.132.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Readiness probe failed: Get "http://10.134.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Readiness probe error: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Readiness probe error: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Readiness probe failed: Get "http://10.134.0.65:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Readiness probe error: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Readiness probe failed: Get "http://10.133.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Readiness probe failed: Get "http://10.133.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Readiness probe failed: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Startup probe failed: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/e2e-ait-e2b0506d-openshift-default Scaled up replica set e2e-ait-e2b0506d-openshift-default-774994d89c from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-ait-e2b0506d-openshift-default-774994d89c Created pod: e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Startup probe failed: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Startup probe failed: Get "http://10.134.0.84:15021/healthz/ready": dial tcp 10.134.0.84:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Startup probe error: Get "http://10.133.0.116:15021/healthz/ready": dial tcp 10.133.0.116:15021: connect: connection refused... 47m Normal AddedLabel gateway/e2e-ait-e2b0506d Added label istio.io/rev=openshift-gateway to gateway e2e-ait-e2b0506d 47m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Startup probe failed: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Startup probe error: Get "http://10.133.0.118:15021/healthz/ready": dial tcp 10.133.0.118:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Startup probe error: Get "http://10.133.0.117:15021/healthz/ready": dial tcp 10.133.0.117:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Startup probe error: Get "http://10.134.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal AddedInterface pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Add eth0 [10.133.0.120/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-derive-93fe9de7-openshift-default-b46c945d9 Created pod: e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r 47m Normal ScalingReplicaSet deployment/e2e-derive-93fe9de7-openshift-default Scaled up replica set e2e-derive-93fe9de7-openshift-default-b46c945d9 from 0 to 1 47m Normal AddedLabel gateway/e2e-derive-93fe9de7 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-93fe9de7 47m Normal Pulled pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Container created 47m Normal Started pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Container started 47m Normal AddedInterface pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Add eth0 [10.133.0.122/23] from ovn-kubernetes 47m Normal AddedLabel gateway/e2e-ait-adca906b Added label istio.io/rev=openshift-gateway to gateway e2e-ait-adca906b 47m Normal SuccessfulCreate replicaset/e2e-ait-adca906b-openshift-default-6dc6cc78d5 Created pod: e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt 47m Normal ScalingReplicaSet deployment/e2e-ait-adca906b-openshift-default Scaled up replica set e2e-ait-adca906b-openshift-default-6dc6cc78d5 from 0 to 1 47m Normal AddedInterface pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Add eth0 [10.134.0.85/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-e2b0506d-655c4bdfdb Created pod: payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559 Error creating: pods "payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e2b0506d: serviceaccount "payload-processing-e2e-ait-e2b0506d" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-e2b0506d Scaled up replica set payload-processing-e2e-ait-e2b0506d-655c4bdfdb from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-e2b0506d Scaled up replica set payload-pre-processing-e2e-ait-e2b0506d-7544d9c559 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-ait-e2b0506d-655c4bdfdb Error creating: pods "payload-processing-e2e-ait-e2b0506d-655c4bdfdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-e2b0506d: serviceaccount "payload-processing-e2e-ait-e2b0506d" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559 Created pod: payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw 47m Normal AddedInterface pod/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk Add eth0 [10.134.0.87/23] from ovn-kubernetes 47m Normal Created pod/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk Container created 47m Normal Pulled pod/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk Container started 47m Normal Created pod/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw Container created 47m Normal Pulled pod/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw Add eth0 [10.134.0.86/23] from ovn-kubernetes 47m Warning FailedCreate replicaset/payload-processing-e2e-derive-93fe9de7-55ffd7cd84 Error creating: pods "payload-processing-e2e-derive-93fe9de7-55ffd7cd84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-93fe9de7: serviceaccount "payload-processing-e2e-derive-93fe9de7" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-93fe9de7-6b46855855 Created pod: payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-93fe9de7 Scaled up replica set payload-pre-processing-e2e-derive-93fe9de7-6b46855855 from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-93fe9de7-6b46855855 Error creating: pods "payload-pre-processing-e2e-derive-93fe9de7-6b46855855-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-93fe9de7: serviceaccount "payload-processing-e2e-derive-93fe9de7" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-93fe9de7-55ffd7cd84 Created pod: payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-93fe9de7 Scaled up replica set payload-processing-e2e-derive-93fe9de7-55ffd7cd84 from 0 to 1 47m Normal Created pod/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 Container created 47m Normal Started pod/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 Container started 47m Normal Created pod/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 Container created 47m Normal Pulled pod/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 Add eth0 [10.134.0.88/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 Add eth0 [10.132.0.83/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 Container started 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-adca906b-565c86788b Created pod: payload-processing-e2e-ait-adca906b-565c86788b-klv42 47m Warning FailedCreate replicaset/payload-processing-e2e-ait-adca906b-565c86788b Error creating: pods "payload-processing-e2e-ait-adca906b-565c86788b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-adca906b: serviceaccount "payload-processing-e2e-ait-adca906b" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-adca906b Scaled up replica set payload-pre-processing-e2e-ait-adca906b-f99ff57d7 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-adca906b-f99ff57d7 Created pod: payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-adca906b-f99ff57d7 Error creating: pods "payload-pre-processing-e2e-ait-adca906b-f99ff57d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-adca906b: serviceaccount "payload-processing-e2e-ait-adca906b" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-adca906b Scaled up replica set payload-processing-e2e-ait-adca906b-565c86788b from 0 to 1 47m Normal Pulled pod/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s Add eth0 [10.132.0.84/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-processing-e2e-ait-adca906b-565c86788b-klv42 Add eth0 [10.134.0.89/23] from ovn-kubernetes 47m Normal Created pod/payload-processing-e2e-ait-adca906b-565c86788b-klv42 Container created 47m Normal Pulled pod/payload-processing-e2e-ait-adca906b-565c86788b-klv42 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-v5g8v 47m Normal Created pod/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s Container created 47m Normal Started pod/payload-processing-e2e-ait-adca906b-565c86788b-klv42 Container started 47m Normal Started pod/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s Container started 47m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-v5g8v Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p 47m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled up replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 4 to 7 47m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Container started 47m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Container created 47m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Container started 47m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Add eth0 [10.132.0.85/23] from ovn-kubernetes 47m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Container created 47m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Container created 47m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Container started 47m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Add eth0 [10.133.0.124/23] from ovn-kubernetes 47m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Add eth0 [10.134.0.90/23] from ovn-kubernetes 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-93fe9de7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-93fe9de7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 47m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Readiness probe error: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Readiness probe error: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Readiness probe failed: Get "http://10.134.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Readiness probe error: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Readiness probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Readiness probe failed: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Readiness probe failed: Get "http://10.132.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Readiness probe failed: Get "http://10.133.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Readiness probe error: Get "http://10.133.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Readiness probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Readiness probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Readiness probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Readiness probe error: Get "http://10.133.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Startup probe failed: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-93fe9de7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-93fe9de7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-adca906b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-adca906b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled up replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 7 to 10 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 (combined from similar events): Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm 46m Normal SuccessfulCreate replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Created pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 46m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Container created 46m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Container created 46m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Container started 46m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Add eth0 [10.132.0.87/23] from ovn-kubernetes 46m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Add eth0 [10.133.0.134/23] from ovn-kubernetes 46m Normal Created pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Container created 46m Normal Pulled pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Container started 46m Normal Started pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Container started 46m Normal AddedInterface pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Add eth0 [10.134.0.96/23] from ovn-kubernetes 46m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Startup probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Startup probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused 46m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused... 46m Normal Killing pod/payload-processing-e2e-ait-e2b0506d-655c4bdfdb-kx6mk Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-ait-e2b0506d-7544d9c559-n49vw Stopping container payload-pre-processing 46m Warning ProbeError pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Readiness probe error: Get "http://10.133.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Readiness probe failed: Get "http://10.133.0.120:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Pulled pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Container created 46m Normal Started pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Container started 46m Normal Killing pod/payload-processing-e2e-derive-93fe9de7-55ffd7cd84-ms4k4 Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-derive-93fe9de7-6b46855855-l6d45 Stopping container payload-pre-processing 46m Normal Created pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Container created 46m Normal Started pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Container started 46m Normal Pulled pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-e2b0506d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-e2b0506d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Startup probe error: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Startup probe failed: Get "http://10.134.0.85:15021/healthz/ready": dial tcp 10.134.0.85:15021: connect: connection refused 46m Normal Killing pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Stopping container istio-proxy 46m Warning ProbeError pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-ait-e2b0506d-openshift-default-774994d89c-hjf9x Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 46m Normal ScalingReplicaSet deployment/e2e-migrate-9ca6fbc2-gw-openshift-default Scaled up replica set e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885 Created pod: e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b 46m Normal AddedLabel gateway/e2e-migrate-9ca6fbc2-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-9ca6fbc2-gw 46m Normal AddedInterface pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Add eth0 [10.133.0.135/23] from ovn-kubernetes 46m Normal Killing pod/payload-pre-processing-e2e-ait-adca906b-f99ff57d7-nnq2s Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-ait-adca906b-565c86788b-klv42 Stopping container payload-processing 46m Normal Killing pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Stopping container istio-proxy 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-adca906b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-adca906b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-9ca6fbc2-9569bb956 Created pod: payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-9ca6fbc2 Scaled up replica set payload-processing-e2e-migrate-9ca6fbc2-9569bb956 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-9ca6fbc2 Scaled up replica set payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87 from 0 to 1 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9245d1-openshift-default Scaled up replica set e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 from 0 to 1 46m Normal AddedLabel gateway/e2e-shared-a-w4-9245d1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-9245d1 46m Warning Unhealthy pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Readiness probe failed: Get "http://10.133.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87 Error creating: pods "payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-9ca6fbc2: serviceaccount "payload-processing-e2e-migrate-9ca6fbc2" not found 46m Warning ProbeError pod/e2e-derive-93fe9de7-openshift-default-b46c945d9-q6v2r Readiness probe error: Get "http://10.133.0.122:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning FailedCreate replicaset/payload-processing-e2e-migrate-9ca6fbc2-9569bb956 Error creating: pods "payload-processing-e2e-migrate-9ca6fbc2-9569bb956-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-9ca6fbc2: serviceaccount "payload-processing-e2e-migrate-9ca6fbc2" not found 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87 Created pod: payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj 45m Normal Started pod/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj Add eth0 [10.134.0.105/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj Add eth0 [10.132.0.88/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj Container created 45m Normal Started pod/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj Container started 45m Normal Created pod/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj Container created 45m Normal Pulled pod/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Add eth0 [10.133.0.136/23] from ovn-kubernetes 45m Normal Killing pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-ait-adca906b-openshift-default-6dc6cc78d5-jzrjt Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-9245d1 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-9245d1 Scaled up replica set payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98 Error creating: pods "payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9245d1: serviceaccount "payload-processing-e2e-shared-a-w4-9245d1" not found 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98 Created pod: payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f Error creating: pods "payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9245d1: serviceaccount "payload-processing-e2e-shared-a-w4-9245d1" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f Created pod: payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt 45m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Container created 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-9ca6fbc2-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-9ca6fbc2-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Add eth0 [10.133.0.138/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Container started 45m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Container started 45m Normal Created pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Container created 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Add eth0 [10.134.0.106/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-6h2wl 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-zprng 45m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Container started 45m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Add eth0 [10.133.0.140/23] from ovn-kubernetes 45m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Add eth0 [10.133.0.141/23] from ovn-kubernetes 45m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Container created 45m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Container started 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Add eth0 [10.134.0.108/23] from ovn-kubernetes 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-mhnwk 45m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Container created 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Container started 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.794s (2.794s including waiting). Image size: 185476477 bytes. 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Startup probe failed: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Startup probe error: Get "http://10.133.0.140:15021/healthz/ready": dial tcp 10.133.0.140:15021: connect: connection refused... 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Startup probe failed: Get "http://10.133.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Startup probe error: Get "http://10.133.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 8 45m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-ngtcs 45m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 45m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Container started 45m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Container started 45m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Add eth0 [10.134.0.109/23] from ovn-kubernetes 45m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Container started 45m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Container created 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Add eth0 [10.132.0.90/23] from ovn-kubernetes 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Add eth0 [10.132.0.89/23] from ovn-kubernetes 45m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Container created 45m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 45m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled up replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 4 to 7 45m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 45m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Add eth0 [10.133.0.143/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Add eth0 [10.134.0.110/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Container created 45m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Container created 45m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Container started 45m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Add eth0 [10.133.0.144/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Container started 45m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Add eth0 [10.133.0.145/23] from ovn-kubernetes 45m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Container created 45m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Container started 45m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Container created 45m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Container started 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Startup probe error: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Startup probe failed: Get "http://10.134.0.110:15021/healthz/ready": dial tcp 10.134.0.110:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Readiness probe failed: Get "http://10.133.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Readiness probe error: Get "http://10.133.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 45m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 45m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 45m Normal Started pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Container started 45m Normal AddedLabel gateway/e2e-shared-b-w4-360818 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-360818 45m Normal Pulled pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Container created 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-360818-openshift-default Scaled up replica set e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f from 0 to 1 45m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Add eth0 [10.132.0.91/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-9ca6fbc2-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-9ca6fbc2-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Killing pod/payload-processing-e2e-migrate-9ca6fbc2-9569bb956-ww7dj Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-migrate-9ca6fbc2-f4894c87-kgcdj Stopping container payload-pre-processing 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-360818 Scaled up replica set payload-processing-e2e-shared-b-w4-360818-848768dc78 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-360818-848768dc78 Error creating: pods "payload-processing-e2e-shared-b-w4-360818-848768dc78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-360818: serviceaccount "payload-processing-e2e-shared-b-w4-360818" not found 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Add eth0 [10.132.0.92/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Container created 45m Warning Unhealthy pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Startup probe failed: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused 45m Warning ProbeError pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Startup probe error: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused... 45m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Add eth0 [10.132.0.93/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f Created pod: payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-360818-848768dc78 Created pod: payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f Error creating: pods "payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-360818: serviceaccount "payload-processing-e2e-shared-b-w4-360818" not found 45m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Container started 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-360818 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f from 0 to 1 45m Normal Created pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Container created 45m Normal Pulled pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Container started 44m Normal Killing pod/e2e-migrate-9ca6fbc2-gw-openshift-default-6cf8844885-4lr9b Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-6b5f19-openshift-default Scaled up replica set e2e-shared-a-w5-6b5f19-openshift-default-85665454c from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-a-w5-6b5f19 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-6b5f19 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-6b5f19-openshift-default-85665454c Created pod: e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d 44m Normal AddedInterface pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Add eth0 [10.133.0.147/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-shared-a-w1-ac7936 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-ac7936 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-ac7936-openshift-default Scaled up replica set e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5 Created pod: e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p 44m Normal AddedInterface pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Add eth0 [10.132.0.94/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-6b5f19 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456 Created pod: payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f Error creating: pods "payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-6b5f19: serviceaccount "payload-processing-e2e-shared-a-w5-6b5f19" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f Created pod: payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456 Error creating: pods "payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-6b5f19: serviceaccount "payload-processing-e2e-shared-a-w5-6b5f19" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-6b5f19 Scaled up replica set payload-processing-e2e-shared-a-w5-6b5f19-cbc474456 from 0 to 1 44m Normal Created pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Add eth0 [10.133.0.150/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Add eth0 [10.133.0.149/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Container started 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479 Created pod: payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-ac7936 Scaled up replica set payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd Error creating: pods "payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-ac7936: serviceaccount "payload-processing-e2e-shared-a-w1-ac7936" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd Created pod: payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-ac7936 Scaled up replica set payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-ac7936: serviceaccount "payload-processing-e2e-shared-a-w1-ac7936" not found 44m Normal Started pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Container started 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Add eth0 [10.132.0.95/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Add eth0 [10.133.0.151/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Container created 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Container created 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Container started 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 44m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 (combined from similar events): Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg 44m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled up replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 7 to 10 44m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 44m Normal SuccessfulCreate replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Created pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z 44m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Container created 44m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Container started 44m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Add eth0 [10.133.0.154/23] from ovn-kubernetes 44m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Add eth0 [10.134.0.111/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Add eth0 [10.132.0.97/23] from ovn-kubernetes 44m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Container started 44m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Container created 44m Normal Started pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Container started 44m Normal Created pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Container created 44m Normal Pulled pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Startup probe error: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Startup probe failed: Get "http://10.134.0.111:15021/healthz/ready": dial tcp 10.134.0.111:15021: connect: connection refused 44m Normal AddedLabel gateway/e2e-mt-be9f2a59 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-be9f2a59 44m Normal ScalingReplicaSet deployment/e2e-mt-be9f2a59-openshift-default Scaled up replica set e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 44m Normal AddedLabel gateway/e2e-shared-a-w3-723ffc Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-723ffc 44m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Add eth0 [10.133.0.155/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-723ffc-openshift-default Scaled up replica set e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf Created pod: e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp 44m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 44m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 44m Normal AddedInterface pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Add eth0 [10.132.0.98/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-be9f2a59 Scaled up replica set payload-processing-e2e-mt-be9f2a59-75b9f5b854 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff Error creating: pods "payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-be9f2a59: serviceaccount "payload-processing-e2e-mt-be9f2a59" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-be9f2a59-75b9f5b854 Created pod: payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff Created pod: payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-be9f2a59 Scaled up replica set payload-pre-processing-e2e-mt-be9f2a59-75665fcbff from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-be9f2a59-75b9f5b854 Error creating: pods "payload-processing-e2e-mt-be9f2a59-75b9f5b854-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-be9f2a59: serviceaccount "payload-processing-e2e-mt-be9f2a59" not found 44m Normal Started pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Container started 44m Normal AddedInterface pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Add eth0 [10.132.0.100/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Add eth0 [10.132.0.99/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Container created 44m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal Pulled pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Container created 44m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal Started pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Container started 44m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-ae1dd9-openshift-default Scaled up replica set e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff from 0 to 1 44m Normal AddedLabel gateway/e2e-shared-b-w5-ae1dd9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-ae1dd9 44m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff Created pod: e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc Error creating: pods "payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-723ffc: serviceaccount "payload-processing-e2e-shared-a-w3-723ffc" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64 Created pod: payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-723ffc Scaled up replica set payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64 Error creating: pods "payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-723ffc: serviceaccount "payload-processing-e2e-shared-a-w3-723ffc" not found 44m Normal AddedInterface pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Add eth0 [10.134.0.112/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc Created pod: payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-723ffc Scaled up replica set payload-processing-e2e-shared-a-w3-723ffc-84c8887d64 from 0 to 1 44m Normal Created pod/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h Container created 44m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h Add eth0 [10.132.0.101/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 Add eth0 [10.133.0.156/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 Container created 44m Normal Started pod/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h Container started 44m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 Container started 43m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Container started 43m Normal Created pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Container created 43m Normal Started pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Container started 43m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Container created 43m Normal Pulled pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedLabel gateway/e2e-shared-b-w1-9d3552 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-9d3552 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-9d3552-openshift-default Scaled up replica set e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 Created pod: e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr 43m Normal AddedInterface pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Add eth0 [10.134.0.113/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-ae1dd9 Scaled up replica set payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-ae1dd9: serviceaccount "payload-processing-e2e-shared-b-w5-ae1dd9" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-ae1dd9 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b Error creating: pods "payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-ae1dd9: serviceaccount "payload-processing-e2e-shared-b-w5-ae1dd9" not found 43m Normal Pulled pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning Unhealthy pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Readiness probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b Created pod: payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm 43m Warning ProbeError pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Readiness probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 43m Normal Created pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Container created 43m Normal Started pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Container started 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95 Created pod: payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Container created 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Add eth0 [10.132.0.102/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 8 to 10 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Add eth0 [10.133.0.157/23] from ovn-kubernetes 43m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Container started 43m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Container created 43m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Add eth0 [10.132.0.103/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Container created 43m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Add eth0 [10.134.0.114/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Container started 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Container started 43m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f (combined from similar events): Created pod: maas-default-gateway-openshift-default-c9ffffc7f-xmv89 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-ac7936-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-ac7936-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning Unhealthy pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 43m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Container started 43m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Container created 43m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-9d3552 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b Error creating: pods "payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-9d3552: serviceaccount "payload-processing-e2e-shared-b-w1-9d3552" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c Created pod: payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-9d3552 Scaled up replica set payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c Error creating: pods "payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-9d3552: serviceaccount "payload-processing-e2e-shared-b-w1-9d3552" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b Created pod: payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz 43m Normal Created pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Container created 43m Normal Pulled pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Add eth0 [10.132.0.105/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Container created 43m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Container started 43m Normal Started pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Add eth0 [10.132.0.104/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Startup probe failed: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused 43m Warning ProbeError pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Startup probe error: Get "http://10.133.0.147:15021/healthz/ready": dial tcp 10.133.0.147:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Startup probe failed: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused 43m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Startup probe error: Get "http://10.134.0.114:15021/healthz/ready": dial tcp 10.134.0.114:15021: connect: connection refused... 43m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Startup probe error: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Startup probe failed: Get "http://10.133.0.157:15021/healthz/ready": dial tcp 10.133.0.157:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-6b5f19-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-6b5f19-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulCreate replicaset/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45 Created pod: e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-ac7936-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-ac7936-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal ScalingReplicaSet deployment/e2e-api-a-d7fde0-openshift-default Scaled up replica set e2e-api-a-d7fde0-openshift-default-7ccbfd9d45 from 0 to 1 43m Normal AddedLabel gateway/e2e-api-a-d7fde0 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d7fde0 43m Normal AddedInterface pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Add eth0 [10.134.0.115/23] from ovn-kubernetes 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d7fde0 Scaled up replica set payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d7fde0 Scaled up replica set payload-processing-e2e-api-a-d7fde0-7f45c4bf74 from 0 to 1 43m Normal Started pod/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj Container started 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f Created pod: payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj Add eth0 [10.132.0.106/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj Add eth0 [10.132.0.107/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d7fde0-7f45c4bf74 Created pod: payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj 43m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d7fde0-7f45c4bf74 Error creating: pods "payload-processing-e2e-api-a-d7fde0-7f45c4bf74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d7fde0: serviceaccount "payload-processing-e2e-api-a-d7fde0" not found 43m Normal Created pod/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj Container created 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f Error creating: pods "payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d7fde0: serviceaccount "payload-processing-e2e-api-a-d7fde0" not found 43m Normal Started pod/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj Container started 43m Normal Created pod/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj Container created 43m Normal Pulled pod/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-6b5f19-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-6b5f19-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal AddedLabel gateway/e2e-mt-7ffb1886 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-7ffb1886 43m Normal ScalingReplicaSet deployment/e2e-mt-7ffb1886-openshift-default Scaled up replica set e2e-mt-7ffb1886-openshift-default-d7d7f6d89 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-mt-7ffb1886-openshift-default-d7d7f6d89 Created pod: e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d 43m Normal AddedInterface pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Add eth0 [10.132.0.109/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-7ffb1886 Scaled up replica set payload-processing-e2e-mt-7ffb1886-64788bf9bd from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-7ffb1886-64788bf9bd Created pod: payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 43m Warning FailedCreate replicaset/payload-processing-e2e-mt-7ffb1886-64788bf9bd Error creating: pods "payload-processing-e2e-mt-7ffb1886-64788bf9bd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7ffb1886: serviceaccount "payload-processing-e2e-mt-7ffb1886" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879 Error creating: pods "payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-7ffb1886: serviceaccount "payload-processing-e2e-mt-7ffb1886" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879 Created pod: payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-7ffb1886 Scaled up replica set payload-pre-processing-e2e-mt-7ffb1886-865ccb8879 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-8e2727-openshift-default-9889b484f Created pod: e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx 43m Normal Pulled pod/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 Container started 43m Normal AddedLabel gateway/e2e-shared-b-w3-8e2727 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-8e2727 43m Normal AddedInterface pod/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 Add eth0 [10.132.0.110/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d Add eth0 [10.133.0.161/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 Container created 43m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-8e2727-openshift-default Scaled up replica set e2e-shared-b-w3-8e2727-openshift-default-9889b484f from 0 to 1 43m Normal Pulled pod/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d Container created 43m Normal Started pod/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d Container started 43m Normal AddedInterface pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Add eth0 [10.133.0.162/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Readiness probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Readiness probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 42m Normal Created pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Container created 42m Normal Pulled pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-723ffc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Started pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Container started 42m Normal Started pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-723ffc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Created pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Container created 42m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Container created 42m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-8e2727 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b Error creating: pods "payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8e2727: serviceaccount "payload-processing-e2e-shared-b-w3-8e2727" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-8e2727 Scaled up replica set payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b from 0 to 1 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-7ffb1886-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-7ffb1886-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c Error creating: pods "payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-8e2727: serviceaccount "payload-processing-e2e-shared-b-w3-8e2727" not found 42m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Container started 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b Created pod: payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c Created pod: payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt 42m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt Add eth0 [10.132.0.111/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 Add eth0 [10.132.0.112/23] from ovn-kubernetes 42m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt Container created 42m Normal Pulled pod/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt Container started 42m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Created pod/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 Container created 42m Warning ProbeError pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 42m Normal Started pod/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 Container started 42m Warning Unhealthy pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 42m Warning ProbeError pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 42m Normal AddedLabel gateway/e2e-api-a-44a8fd Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-44a8fd 42m Normal SuccessfulCreate replicaset/e2e-api-a-44a8fd-openshift-default-5bb9d64b95 Created pod: e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b 42m Normal ScalingReplicaSet deployment/e2e-api-a-44a8fd-openshift-default Scaled up replica set e2e-api-a-44a8fd-openshift-default-5bb9d64b95 from 0 to 1 42m Normal AddedInterface pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Add eth0 [10.134.0.118/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Startup probe error: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Startup probe failed: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-ae1dd9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-ae1dd9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-44a8fd Scaled up replica set payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc Created pod: payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc Error creating: pods "payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-44a8fd: serviceaccount "payload-processing-e2e-api-a-44a8fd" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-api-a-44a8fd-7544b9d8b4 Error creating: pods "payload-processing-e2e-api-a-44a8fd-7544b9d8b4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-44a8fd: serviceaccount "payload-processing-e2e-api-a-44a8fd" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-44a8fd Scaled up replica set payload-processing-e2e-api-a-44a8fd-7544b9d8b4 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-44a8fd-7544b9d8b4 Created pod: payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 42m Normal Created pod/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd Container created 42m Normal Created pod/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd Add eth0 [10.132.0.113/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Pulled pod/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd Container started 42m Normal AddedInterface pod/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 Add eth0 [10.132.0.114/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 Container started 42m Normal ScalingReplicaSet deployment/e2e-inf-a-d30e9f-openshift-default Scaled up replica set e2e-inf-a-d30e9f-openshift-default-776cc5cb65 from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-inf-a-d30e9f-openshift-default-776cc5cb65 Created pod: e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 42m Normal AddedLabel gateway/e2e-inf-a-d30e9f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d30e9f 42m Normal AddedInterface pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Add eth0 [10.134.0.123/23] from ovn-kubernetes 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-ae1dd9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-ae1dd9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546 Created pod: payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779 Error creating: pods "payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d30e9f: serviceaccount "payload-processing-e2e-inf-a-d30e9f" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546 Error creating: pods "payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d30e9f: serviceaccount "payload-processing-e2e-inf-a-d30e9f" not found 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779 Created pod: payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-d30e9f Scaled up replica set payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779 from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-d30e9f Scaled up replica set payload-processing-e2e-inf-a-d30e9f-5f9c9d8546 from 0 to 1 42m Normal Started pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Container started 42m Normal Created pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Container created 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Add eth0 [10.132.0.115/23] from ovn-kubernetes 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Add eth0 [10.133.0.163/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Container created 42m Normal Started pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Container started 42m Normal Pulled pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedLabel gateway/e2e-api-b-c1abc2 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-c1abc2 42m Normal ScalingReplicaSet deployment/e2e-api-b-c1abc2-openshift-default Scaled up replica set e2e-api-b-c1abc2-openshift-default-97b78987f from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-api-b-c1abc2-openshift-default-97b78987f Created pod: e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl 42m Normal AddedInterface pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Add eth0 [10.132.0.116/23] from ovn-kubernetes 42m Warning Unhealthy pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Readiness probe failed: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Readiness probe error: Get "http://10.134.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-44a8fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-44a8fd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-c1abc2-7795df7847 Created pod: payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 42m Normal Pulled pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-c1abc2 Scaled up replica set payload-pre-processing-e2e-api-b-c1abc2-7795df7847 from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-c1abc2-7795df7847 Error creating: pods "payload-pre-processing-e2e-api-b-c1abc2-7795df7847-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-c1abc2: serviceaccount "payload-processing-e2e-api-b-c1abc2" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4 Error creating: pods "payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-c1abc2: serviceaccount "payload-processing-e2e-api-b-c1abc2" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-c1abc2 Scaled up replica set payload-processing-e2e-api-b-c1abc2-75c7bd5cb4 from 0 to 1 42m Normal Created pod/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 Container created 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4 Created pod: payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb 42m Normal Started pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Container started 42m Normal Started pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Container started 42m Normal Created pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Container created 42m Normal Started pod/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 Container started 42m Normal Created pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 Add eth0 [10.134.0.126/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-be9f2a59-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-be9f2a59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d7fde0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d7fde0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc 42m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c Stopping container istio-proxy 42m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-ngdvc Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-7cnf4 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s 42m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Stopping container istio-proxy 42m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9gx7c 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 6; reason: All metrics below target 42m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled down replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 10 to 6 42m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-d4z2s Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal AddedInterface pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb Add eth0 [10.133.0.164/23] from ovn-kubernetes 42m Normal Started pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb Container started 42m Normal Created pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb Container created 42m Normal Pulled pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d7fde0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d7fde0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Readiness probe failed: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Readiness probe error: Get "http://10.132.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Pulled pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Container started 41m Normal Created pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Container created 41m Normal ScalingReplicaSet deployment/e2e-api-a-a3337a-openshift-default Scaled up replica set e2e-api-a-a3337a-openshift-default-6c5ddf5bdf from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf Created pod: e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 41m Normal AddedLabel gateway/e2e-api-a-a3337a Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-a3337a 41m Normal AddedInterface pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Add eth0 [10.133.0.168/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Startup probe failed: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused 41m Warning ProbeError pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Startup probe error: Get "http://10.132.0.109:15021/healthz/ready": dial tcp 10.132.0.109:15021: connect: connection refused... 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-a3337a-679664c6d7 Error creating: pods "payload-pre-processing-e2e-api-a-a3337a-679664c6d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a3337a: serviceaccount "payload-processing-e2e-api-a-a3337a" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-a3337a-cfbc4f494 Created pod: payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-a3337a Scaled up replica set payload-processing-e2e-api-a-a3337a-cfbc4f494 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-a-a3337a-cfbc4f494 Error creating: pods "payload-processing-e2e-api-a-a3337a-cfbc4f494-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a3337a: serviceaccount "payload-processing-e2e-api-a-a3337a" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-a3337a-679664c6d7 Created pod: payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-a3337a Scaled up replica set payload-pre-processing-e2e-api-a-a3337a-679664c6d7 from 0 to 1 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc Add eth0 [10.133.0.169/23] from ovn-kubernetes 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Started pod/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc Container started 41m Normal Pulled pod/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc Container created 41m Normal Killing pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Stopping container payload-pre-processing 41m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled down replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 10 to 6 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-gjw7j 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl Stopping container istio-proxy 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled down replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 10 to 6 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-7wxgn 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-9hsjl 41m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv 41m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rrr2p Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-w5qzv Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 41m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled down replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 10 to 6 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 6; reason: All metrics below target 41m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-pnzg2 Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-8gd6q 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j 41m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-djd22 41m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-xm26j Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jl4md Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-jcf5f Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-kptwp Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal Pulled pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr Container started 41m Normal AddedInterface pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr Add eth0 [10.133.0.170/23] from ovn-kubernetes 41m Normal Created pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr Container created 41m Normal ScalingReplicaSet deployment/e2e-mt-be9f2a59-openshift-default Scaled up replica set e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 from 1 to 2 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-be9f2a59-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Add eth0 [10.133.0.172/23] from ovn-kubernetes 41m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Container started 41m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 41m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Container created 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-723ffc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-723ffc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-7ffb1886-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-7ffb1886-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning Unhealthy pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-825bc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Normal AddedLabel gateway/e2e-api-b-ab09ae Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-ab09ae 41m Normal SuccessfulCreate replicaset/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f Created pod: e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf 41m Normal ScalingReplicaSet deployment/e2e-api-b-ab09ae-openshift-default Scaled up replica set e2e-api-b-ab09ae-openshift-default-5dcb9bb99f from 0 to 1 41m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused 41m Warning Unhealthy pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-5zlvp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 41m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused... 41m Normal AddedInterface pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Add eth0 [10.133.0.173/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-ab09ae Scaled up replica set payload-processing-e2e-api-b-ab09ae-5968dfbd96 from 0 to 1 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5 Error creating: pods "payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ab09ae: serviceaccount "payload-processing-e2e-api-b-ab09ae" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-ab09ae Scaled up replica set payload-pre-processing-e2e-api-b-ab09ae-789ff776f5 from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-api-b-ab09ae-5968dfbd96 Error creating: pods "payload-processing-e2e-api-b-ab09ae-5968dfbd96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-ab09ae: serviceaccount "payload-processing-e2e-api-b-ab09ae" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5 Created pod: payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb 41m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb Add eth0 [10.133.0.174/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-ab09ae-5968dfbd96 Created pod: payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j 41m Normal Pulled pod/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb Container started 41m Normal Started pod/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j Container started 41m Normal Created pod/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j Container created 41m Normal AddedInterface pod/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j Add eth0 [10.133.0.175/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb Container created 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Readiness probe failed: Get "http://10.133.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Readiness probe error: Get "http://10.133.0.143:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning ProbeError pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Readiness probe error: Get "http://10.132.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Readiness probe failed: Get "http://10.132.0.116:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal AddedLabel gateway/e2e-inf-b-cea203 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-cea203 41m Normal ScalingReplicaSet deployment/e2e-inf-b-cea203-openshift-default Scaled up replica set e2e-inf-b-cea203-openshift-default-68cc5c94f7 from 0 to 1 41m Normal SuccessfulCreate replicaset/e2e-inf-b-cea203-openshift-default-68cc5c94f7 Created pod: e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn 41m Normal AddedInterface pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Add eth0 [10.133.0.177/23] from ovn-kubernetes 41m Normal Created pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Container created 41m Normal Started pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Container started 41m Warning ProbeError pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Readiness probe error: Get "http://10.134.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal Pulled pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning Unhealthy pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Readiness probe failed: Get "http://10.134.0.123:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal Started pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Container started 41m Warning ProbeError pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Readiness probe error: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Readiness probe failed: Get "http://10.134.0.118:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Normal Pulled pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Container created 40m Normal Started pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Container started 40m Normal Created pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Container created 40m Normal Started pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Container started 40m Normal Pulled pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Container created 40m Normal Pulled pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-cea203 Scaled up replica set payload-pre-processing-e2e-inf-b-cea203-5779bf5c55 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55 Created pod: payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-cea203 Scaled up replica set payload-processing-e2e-inf-b-cea203-f4cc9fbcd from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-cea203-f4cc9fbcd Created pod: payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55 Error creating: pods "payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-cea203: serviceaccount "payload-processing-e2e-inf-b-cea203" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-cea203-f4cc9fbcd Error creating: pods "payload-processing-e2e-inf-b-cea203-f4cc9fbcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-cea203: serviceaccount "payload-processing-e2e-inf-b-cea203" not found 40m Normal Pulled pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Add eth0 [10.133.0.178/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Container created 40m Normal AddedInterface pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Add eth0 [10.133.0.179/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 40m Normal Started pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Container started 40m Normal Created pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Container created 40m Normal Started pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Container started 40m Warning Unhealthy pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Startup probe failed: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused 40m Warning ProbeError pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Startup probe error: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Startup probe failed: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Startup probe error: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Startup probe failed: Get "http://10.134.0.123:15021/healthz/ready": dial tcp 10.134.0.123:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Startup probe error: Get "http://10.134.0.118:15021/healthz/ready": dial tcp 10.134.0.118:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p 40m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled down replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 10 to 6 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln 40m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-c1abc2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-qhdhw Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-rcwln Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-lvtr9 Stopping container istio-proxy 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-c1abc2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8nc5p Stopping container istio-proxy 40m Normal Killing pod/payload-pre-processing-e2e-api-a-d7fde0-857bc97f7f-4hssj Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-api-a-d7fde0-7f45c4bf74-4zggj Stopping container payload-processing 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled down replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 10 to 6 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-c1abc2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr 40m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-whbbr Stopping container istio-proxy 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-c1abc2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Stopping container istio-proxy 40m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-9cmqz Stopping container istio-proxy 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-cea203-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-cea203-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Readiness probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8rd4b Readiness probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 40m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Readiness probe error: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-z9xgv Readiness probe failed: Get "http://10.133.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d30e9f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d30e9f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-8e2727-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Readiness probe failed: Get "http://10.133.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-8e2727-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Readiness probe error: Get "http://10.133.0.168:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-be9f2a59-75b9f5b854 Created pod: payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr 40m Warning FailedCreate replicaset/payload-processing-e2e-mt-be9f2a59-75b9f5b854 Error creating: pods "payload-processing-e2e-mt-be9f2a59-75b9f5b854-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-be9f2a59: serviceaccount "payload-processing-e2e-mt-be9f2a59" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-be9f2a59 Scaled up replica set payload-pre-processing-e2e-mt-be9f2a59-75665fcbff from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-be9f2a59 Scaled up replica set payload-processing-e2e-mt-be9f2a59-75b9f5b854 from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff Created pod: payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff Error creating: pods "payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-be9f2a59: serviceaccount "payload-processing-e2e-mt-be9f2a59" not found 40m Normal AddedInterface pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 Add eth0 [10.133.0.182/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Container created 40m Normal Pulled pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Container started 40m Normal Started pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 Container started 40m Normal AddedInterface pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr Add eth0 [10.133.0.183/23] from ovn-kubernetes 40m Normal Created pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 Container created 40m Normal Pulled pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr Container created 40m Normal Started pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr Container started 40m Normal SuccessfulCreate replicaset/e2e-api-b-71f552-openshift-default-665776fb8c Created pod: e2e-api-b-71f552-openshift-default-665776fb8c-8j94q 40m Normal AddedLabel gateway/e2e-api-b-71f552 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-71f552 40m Normal ScalingReplicaSet deployment/e2e-api-b-71f552-openshift-default Scaled up replica set e2e-api-b-71f552-openshift-default-665776fb8c from 0 to 1 40m Normal AddedInterface pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Add eth0 [10.133.0.184/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Startup probe failed: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Startup probe error: Get "http://10.133.0.168:15021/healthz/ready": dial tcp 10.133.0.168:15021: connect: connection refused... 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-71f552 Scaled up replica set payload-processing-e2e-api-b-71f552-fd4bfdd56 from 0 to 1 40m Warning FailedCreate replicaset/payload-processing-e2e-api-b-71f552-fd4bfdd56 Error creating: pods "payload-processing-e2e-api-b-71f552-fd4bfdd56-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-71f552: serviceaccount "payload-processing-e2e-api-b-71f552" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-71f552 Scaled up replica set payload-pre-processing-e2e-api-b-71f552-55789dbff from 0 to 1 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-71f552-55789dbff Error creating: pods "payload-pre-processing-e2e-api-b-71f552-55789dbff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-71f552: serviceaccount "payload-processing-e2e-api-b-71f552" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-71f552-fd4bfdd56 Created pod: payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 40m Normal AddedInterface pod/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 Add eth0 [10.133.0.185/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-71f552-55789dbff Created pod: payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl 40m Normal Started pod/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 Container started 40m Normal Created pod/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl Container created 40m Normal Created pod/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 Container created 40m Normal Started pod/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl Container started 40m Normal Pulled pod/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a3337a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl Add eth0 [10.133.0.186/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a3337a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-44a8fd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-44a8fd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Killing pod/payload-pre-processing-e2e-mt-7ffb1886-865ccb8879-d8h6d Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-mt-7ffb1886-64788bf9bd-5zrf4 Stopping container payload-processing 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a3337a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a3337a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning Unhealthy pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Readiness probe failed: Get "http://10.133.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Readiness probe error: Get "http://10.133.0.177:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Normal Created pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Container created 39m Normal Started pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Container started 39m Normal Pulled pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Container created 39m Normal Started pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Container started 39m Warning Unhealthy pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused 39m Warning ProbeError pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 39m Normal Killing pod/payload-pre-processing-e2e-api-a-44a8fd-7dc6bc45bc-kd9hd Stopping container payload-pre-processing 39m Normal Killing pod/payload-processing-e2e-api-a-44a8fd-7544b9d8b4-d79n8 Stopping container payload-processing 39m Normal Killing pod/e2e-mt-7ffb1886-openshift-default-d7d7f6d89-lbd9d Stopping container istio-proxy 39m Normal AddedLabel gateway/e2e-api-a-67440d Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-67440d 39m Normal ScalingReplicaSet deployment/e2e-api-a-67440d-openshift-default Scaled up replica set e2e-api-a-67440d-openshift-default-5584bd5bc from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-api-a-67440d-openshift-default-5584bd5bc Created pod: e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b 39m Normal AddedInterface pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Add eth0 [10.133.0.188/23] from ovn-kubernetes 39m Normal Killing pod/e2e-api-a-d7fde0-openshift-default-7ccbfd9d45-6s7v7 Stopping container istio-proxy 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-cea203-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-cea203-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-67440d-547489cbdc Created pod: payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j 39m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-67440d-547489cbdc Error creating: pods "payload-pre-processing-e2e-api-a-67440d-547489cbdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-67440d: serviceaccount "payload-processing-e2e-api-a-67440d" not found 39m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-67440d-766b4cb948 Created pod: payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d 39m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-67440d Scaled up replica set payload-processing-e2e-api-a-67440d-766b4cb948 from 0 to 1 39m Warning FailedCreate replicaset/payload-processing-e2e-api-a-67440d-766b4cb948 Error creating: pods "payload-processing-e2e-api-a-67440d-766b4cb948-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-67440d: serviceaccount "payload-processing-e2e-api-a-67440d" not found 39m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-67440d Scaled up replica set payload-pre-processing-e2e-api-a-67440d-547489cbdc from 0 to 1 39m Normal Pulled pod/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Started pod/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d Container started 39m Normal AddedInterface pod/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d Add eth0 [10.133.0.191/23] from ovn-kubernetes 39m Normal Created pod/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j Container created 39m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j Add eth0 [10.133.0.192/23] from ovn-kubernetes 39m Normal Started pod/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j Container started 39m Normal Created pod/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d Container created 39m Normal Pulled pod/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 39m Normal Killing pod/payload-processing-e2e-api-b-c1abc2-75c7bd5cb4-kqvsb Stopping container payload-processing 39m Normal Killing pod/payload-pre-processing-e2e-api-b-c1abc2-7795df7847-j9896 Stopping container payload-pre-processing 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled down replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 10 to 8 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 8; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-ab09ae-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-k8qd5 Stopping container istio-proxy 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-ab09ae-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-lsqc9 Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled down replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 8 to 7 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z 38m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-bhs4z Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-zn4gg Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qlwdb 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled down replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 6 to 5 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 7; reason: All metrics below target 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled down replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 6 to 5 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm 38m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled down replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 6 to 5 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 5; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Stopping container istio-proxy 38m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Readiness probe failed: HTTP probe failed with statuscode: 503 38m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5q5lm Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal ScalingReplicaSet deployment/e2e-mt-be9f2a59-openshift-default Scaled up replica set e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 from 2 to 3 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-be9f2a59-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Add eth0 [10.134.0.135/23] from ovn-kubernetes 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Container created 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Container started 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused 38m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-8547m Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled down replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 6 to 5 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 38m Normal Pulled pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-be9f2a59-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 38m Normal ScalingReplicaSet deployment/e2e-mt-be9f2a59-openshift-default Scaled up replica set e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 from 3 to 6 38m Normal Started pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Container started 38m Normal Created pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Container created 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Container created 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Container created 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Container started 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Add eth0 [10.132.0.117/23] from ovn-kubernetes 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Container started 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Container created 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Add eth0 [10.134.0.138/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Add eth0 [10.132.0.118/23] from ovn-kubernetes 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Container started 38m Normal Killing pod/payload-pre-processing-e2e-mt-be9f2a59-75665fcbff-w7dn9 Stopping container payload-pre-processing 38m Normal Killing pod/payload-processing-e2e-mt-be9f2a59-75b9f5b854-v9fcr Stopping container payload-processing 38m Warning Unhealthy pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Startup probe failed: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused 38m Warning ProbeError pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Startup probe error: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Startup probe error: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Startup probe error: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Startup probe failed: Get "http://10.132.0.118:15021/healthz/ready": dial tcp 10.132.0.118:15021: connect: connection refused 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Startup probe failed: Get "http://10.132.0.117:15021/healthz/ready": dial tcp 10.132.0.117:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c 38m Normal SuccessfulCreate replicaset/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 (combined from similar events): Created pod: e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-be9f2a59-openshift-default New size: 10; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-mt-be9f2a59-openshift-default Scaled up replica set e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9 from 6 to 10 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Container started 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Container started 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Container created 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Add eth0 [10.133.0.196/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Container created 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Container started 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Add eth0 [10.133.0.195/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Add eth0 [10.132.0.119/23] from ovn-kubernetes 38m Normal Killing pod/payload-processing-e2e-api-a-a3337a-cfbc4f494-z9pkr Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-api-a-a3337a-679664c6d7-gb2jc Stopping container payload-pre-processing 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Container created 38m Normal AddedInterface pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Add eth0 [10.134.0.140/23] from ovn-kubernetes 38m Normal Created pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Container created 38m Normal Started pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Container started 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Pulled pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-71f552-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-71f552-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulCreate replicaset/e2e-inf-a-16df10-openshift-default-6d7dffd555 Created pod: e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r 38m Normal AddedLabel gateway/e2e-inf-a-16df10 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-16df10 38m Normal ScalingReplicaSet deployment/e2e-inf-a-16df10-openshift-default Scaled up replica set e2e-inf-a-16df10-openshift-default-6d7dffd555 from 0 to 1 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Startup probe error: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Startup probe failed: Get "http://10.133.0.195:15021/healthz/ready": dial tcp 10.133.0.195:15021: connect: connection refused 38m Normal AddedInterface pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Add eth0 [10.133.0.197/23] from ovn-kubernetes 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 38m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Startup probe error: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Startup probe failed: Get "http://10.133.0.196:15021/healthz/ready": dial tcp 10.133.0.196:15021: connect: connection refused 38m Normal Killing pod/e2e-api-a-44a8fd-openshift-default-5bb9d64b95-8mv2b Stopping container istio-proxy 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-16df10-859bf4d79 Created pod: payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-16df10 Scaled up replica set payload-processing-e2e-inf-a-16df10-58fb5c86cb from 0 to 1 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-16df10-859bf4d79 Error creating: pods "payload-pre-processing-e2e-inf-a-16df10-859bf4d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-16df10: serviceaccount "payload-processing-e2e-inf-a-16df10" not found 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-16df10-58fb5c86cb Error creating: pods "payload-processing-e2e-inf-a-16df10-58fb5c86cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-16df10: serviceaccount "payload-processing-e2e-inf-a-16df10" not found 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-16df10 Scaled up replica set payload-pre-processing-e2e-inf-a-16df10-859bf4d79 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-16df10-58fb5c86cb Created pod: payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr 38m Normal Created pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Container created 38m Normal Started pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Container started 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Add eth0 [10.133.0.198/23] from ovn-kubernetes 38m Normal Pulled pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Container created 38m Normal Pulled pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Container started 38m Normal AddedInterface pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Add eth0 [10.133.0.199/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-71f552-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-71f552-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/payload-pre-processing-e2e-api-b-ab09ae-789ff776f5-nstcb Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-api-b-ab09ae-5968dfbd96-5ck5j Stopping container payload-processing 37m Warning Unhealthy pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Stopping container istio-proxy 37m Warning ProbeError pod/e2e-api-b-c1abc2-openshift-default-97b78987f-mq5wl Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal ScalingReplicaSet deployment/e2e-inf-a-c1fce3-openshift-default Scaled up replica set e2e-inf-a-c1fce3-openshift-default-7f6c4d6669 from 0 to 1 37m Normal AddedLabel gateway/e2e-inf-a-c1fce3 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-c1fce3 37m Normal SuccessfulCreate replicaset/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669 Created pod: e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm 37m Normal AddedInterface pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Add eth0 [10.133.0.200/23] from ovn-kubernetes 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-16df10-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-16df10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal SuccessfulCreate replicaset/e2e-api-b-734d43-openshift-default-65fd65866f Created pod: e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b 37m Normal ScalingReplicaSet deployment/e2e-api-b-734d43-openshift-default Scaled up replica set e2e-api-b-734d43-openshift-default-65fd65866f from 0 to 1 37m Normal AddedLabel gateway/e2e-api-b-734d43 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-734d43 37m Warning ProbeError pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Readiness probe error: Get "http://10.133.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 37m Warning Unhealthy pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Readiness probe failed: Get "http://10.133.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 37m Normal AddedInterface pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Add eth0 [10.133.0.201/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Started pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Container started 37m Normal Created pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Container created 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b Created pod: payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47 Created pod: payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-c1fce3 Scaled up replica set payload-processing-e2e-inf-a-c1fce3-bcbf85d6b from 0 to 1 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b Error creating: pods "payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c1fce3: serviceaccount "payload-processing-e2e-inf-a-c1fce3" not found 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47 Error creating: pods "payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-c1fce3: serviceaccount "payload-processing-e2e-inf-a-c1fce3" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-c1fce3 Scaled up replica set payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47 from 0 to 1 37m Normal Pulled pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Add eth0 [10.133.0.202/23] from ovn-kubernetes 37m Normal AddedInterface pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Add eth0 [10.133.0.203/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Container created 37m Normal Started pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Container started 37m Normal Created pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Container created 37m Normal Started pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Container started 37m Warning FailedCreate replicaset/payload-processing-e2e-api-b-734d43-f88dd95f4 Error creating: pods "payload-processing-e2e-api-b-734d43-f88dd95f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-734d43: serviceaccount "payload-processing-e2e-api-b-734d43" not found 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-734d43 Scaled up replica set payload-processing-e2e-api-b-734d43-f88dd95f4 from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-734d43-67ff74f87f Error creating: pods "payload-pre-processing-e2e-api-b-734d43-67ff74f87f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-734d43: serviceaccount "payload-processing-e2e-api-b-734d43" not found 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-734d43-67ff74f87f Created pod: payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-734d43 Scaled up replica set payload-pre-processing-e2e-api-b-734d43-67ff74f87f from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-734d43-f88dd95f4 Created pod: payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp 37m Normal Pulled pod/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k Add eth0 [10.133.0.204/23] from ovn-kubernetes 37m Warning ProbeError pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Startup probe error: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Startup probe failed: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused 37m Normal AddedInterface pod/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp Add eth0 [10.133.0.205/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k Container created 37m Normal Created pod/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp Container created 37m Normal Started pod/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k Container started 37m Normal Started pod/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp Container started 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-67440d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-67440d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c1fce3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c1fce3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-67440d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-67440d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled down replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 4; reason: All metrics below target 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Stopping container istio-proxy 37m Normal Killing pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-n2pf5 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled down replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl 37m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled down replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 6 to 4 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 37m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-wrrv4 Stopping container istio-proxy 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-wrjk6 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 5; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled down replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 6 to 5 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-752j4 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default Scaled down replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 5 to 4 37m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-lsbkl Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 4; reason: All metrics below target 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp 37m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-s76zp Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-mrwb7 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 4; reason: All metrics below target 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-c4c85 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-6rv2c Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-52nxf Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-zc9cs Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-rhg9v Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-4zrn6 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-62k4s Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning ProbeError pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Readiness probe error: HTTP probe failed with statuscode: 503... 37m Warning Unhealthy pod/e2e-mt-be9f2a59-openshift-default-6b7dcc8bb9-gkr75 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-qcdp7 Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal Killing pod/e2e-api-a-a3337a-openshift-default-6c5ddf5bdf-mcsw7 Stopping container istio-proxy 36m Normal Killing pod/payload-pre-processing-e2e-api-b-71f552-55789dbff-bksnl Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-api-b-71f552-fd4bfdd56-vw4k8 Stopping container payload-processing 36m Normal AddedLabel gateway/e2e-inf-b-537001 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-537001 36m Normal SuccessfulCreate replicaset/e2e-inf-b-537001-openshift-default-74f94b549 Created pod: e2e-inf-b-537001-openshift-default-74f94b549-kh7mf 36m Normal ScalingReplicaSet deployment/e2e-inf-b-537001-openshift-default Scaled up replica set e2e-inf-b-537001-openshift-default-74f94b549 from 0 to 1 36m Normal AddedInterface pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Add eth0 [10.133.0.209/23] from ovn-kubernetes 36m Normal Killing pod/e2e-api-b-ab09ae-openshift-default-5dcb9bb99f-pq7vf Stopping container istio-proxy 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-537001-66fd974b46 Error creating: pods "payload-processing-e2e-inf-b-537001-66fd974b46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-537001: serviceaccount "payload-processing-e2e-inf-b-537001" not found 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-537001-554c9c6887 Error creating: pods "payload-pre-processing-e2e-inf-b-537001-554c9c6887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-537001: serviceaccount "payload-processing-e2e-inf-b-537001" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-537001 Scaled up replica set payload-processing-e2e-inf-b-537001-66fd974b46 from 0 to 1 36m Normal AddedLabel gateway/e2e-inf-a-e7f767 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-e7f767 36m Normal ScalingReplicaSet deployment/e2e-inf-a-e7f767-openshift-default Scaled up replica set e2e-inf-a-e7f767-openshift-default-7d54d7f747 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-inf-a-e7f767-openshift-default-7d54d7f747 Created pod: e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-537001 Scaled up replica set payload-pre-processing-e2e-inf-b-537001-554c9c6887 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-537001-66fd974b46 Created pod: payload-processing-e2e-inf-b-537001-66fd974b46-cbngf 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-537001-554c9c6887 Created pod: payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c 36m Normal Created pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Container created 36m Normal AddedInterface pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Add eth0 [10.133.0.210/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Add eth0 [10.133.0.211/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Container started 36m Normal Pulled pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Add eth0 [10.132.0.120/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Container started 36m Normal Created pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Container created 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-e7f767 Scaled up replica set payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-e7f767-844d6b5bdd Created pod: payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559 Created pod: payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559 Error creating: pods "payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e7f767: serviceaccount "payload-processing-e2e-inf-a-e7f767" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-e7f767 Scaled up replica set payload-processing-e2e-inf-a-e7f767-844d6b5bdd from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-e7f767-844d6b5bdd Error creating: pods "payload-processing-e2e-inf-a-e7f767-844d6b5bdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e7f767: serviceaccount "payload-processing-e2e-inf-a-e7f767" not found 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Add eth0 [10.133.0.212/23] from ovn-kubernetes 36m Normal Started pod/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 Container started 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 Add eth0 [10.133.0.213/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Container started 36m Normal Created pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Container created 36m Normal Pulled pod/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 Container created 36m Warning ProbeError pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Readiness probe error: Get "http://10.133.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Readiness probe failed: Get "http://10.133.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal Started pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Container started 36m Normal Pulled pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Container started 36m Normal Created pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Container created 36m Normal Created pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Container created 36m Normal Pulled pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Startup probe failed: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused 36m Warning Unhealthy pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Startup probe failed: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused 36m Warning ProbeError pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Startup probe error: Get "http://10.133.0.201:15021/healthz/ready": dial tcp 10.133.0.201:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Startup probe error: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused... 36m Normal ScalingReplicaSet deployment/e2e-inf-b-4f415d-openshift-default Scaled up replica set e2e-inf-b-4f415d-openshift-default-549bd7f477 from 0 to 1 36m Normal AddedLabel gateway/e2e-inf-b-4f415d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4f415d 36m Normal SuccessfulCreate replicaset/e2e-inf-b-4f415d-openshift-default-549bd7f477 Created pod: e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 36m Normal AddedInterface pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Add eth0 [10.134.0.146/23] from ovn-kubernetes 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-734d43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-734d43-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Normal SuccessfulCreate replicaset/e2e-inf-b-cea203-openshift-default-68cc5c94f7 Created pod: e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-cea203-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 36m Normal ScalingReplicaSet deployment/e2e-inf-b-cea203-openshift-default Scaled up replica set e2e-inf-b-cea203-openshift-default-68cc5c94f7 from 1 to 2 36m Normal Pulled pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Add eth0 [10.132.0.121/23] from ovn-kubernetes 36m Normal Created pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Container created 36m Normal Started pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Container started 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-4f415d Scaled up replica set payload-processing-e2e-inf-b-4f415d-5d8f4cb6b from 0 to 1 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799 Error creating: pods "payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4f415d: serviceaccount "payload-processing-e2e-inf-b-4f415d" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799 Created pod: payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-4f415d Scaled up replica set payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799 from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b Error creating: pods "payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4f415d: serviceaccount "payload-processing-e2e-inf-b-4f415d" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b Created pod: payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 36m Normal Created pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Container created 36m Normal Pulled pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Add eth0 [10.133.0.214/23] from ovn-kubernetes 36m Normal Pulled pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Container created 36m Normal Started pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Container started 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Add eth0 [10.133.0.215/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Container created 35m Normal Started pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Container started 35m Warning ProbeError pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 35m Normal Killing pod/payload-processing-e2e-api-a-67440d-766b4cb948-jtg5d Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-api-a-67440d-547489cbdc-j8d4j Stopping container payload-pre-processing 35m Warning Unhealthy pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Startup probe failed: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-16df10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-16df10-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning ProbeError pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Startup probe error: Get "http://10.133.0.200:15021/healthz/ready": dial tcp 10.133.0.200:15021: connect: connection refused... 35m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 4; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 35m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-8jqd6 Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 4; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default Scaled down replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 5 to 4 35m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default Scaled down replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 5 to 4 35m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-dnj4p 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-734d43-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-734d43-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Killing pod/e2e-api-b-71f552-openshift-default-665776fb8c-8j94q Stopping container istio-proxy 35m Normal AddedLabel gateway/e2e-inf-a-f21058 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f21058 35m Normal ScalingReplicaSet deployment/e2e-inf-a-f21058-openshift-default Scaled up replica set e2e-inf-a-f21058-openshift-default-57fbbd47dc from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-inf-a-f21058-openshift-default-57fbbd47dc Created pod: e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg 35m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Readiness probe error: Get "http://10.133.0.136:15021/healthz/ready": read tcp 10.133.0.2:47664->10.133.0.136:15021: read: connection reset by peer... 35m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Readiness probe failed: Get "http://10.133.0.136:15021/healthz/ready": read tcp 10.133.0.2:47664->10.133.0.136:15021: read: connection reset by peer 35m Normal AddedInterface pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Add eth0 [10.133.0.217/23] from ovn-kubernetes 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-c1fce3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-c1fce3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Container started 35m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Container created 35m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f21058-68466cc45f Error creating: pods "payload-pre-processing-e2e-inf-a-f21058-68466cc45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f21058: serviceaccount "payload-processing-e2e-inf-a-f21058" not found 35m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-360818-openshift-default Scaled up replica set e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f from 1 to 2 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f21058 Scaled up replica set payload-pre-processing-e2e-inf-a-f21058-68466cc45f from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f21058-86d9b6f68 Error creating: pods "payload-processing-e2e-inf-a-f21058-86d9b6f68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f21058: serviceaccount "payload-processing-e2e-inf-a-f21058" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f21058-68466cc45f Created pod: payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f21058 Scaled up replica set payload-processing-e2e-inf-a-f21058-86d9b6f68 from 0 to 1 35m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Add eth0 [10.133.0.218/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp Container started 35m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Container created 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f21058-86d9b6f68 Created pod: payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp 35m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Container started 35m Normal Started pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Container started 35m Normal Created pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Container created 35m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Add eth0 [10.132.0.123/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp Add eth0 [10.132.0.124/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9245d1-openshift-default Scaled up replica set e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 from 1 to 2 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Startup probe failed: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused 35m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Container created 35m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Container started 35m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Add eth0 [10.132.0.125/23] from ovn-kubernetes 35m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Startup probe error: Get "http://10.133.0.218:15021/healthz/ready": dial tcp 10.133.0.218:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Startup probe error: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Startup probe failed: Get "http://10.132.0.125:15021/healthz/ready": dial tcp 10.132.0.125:15021: connect: connection refused 35m Normal Killing pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Stopping container payload-pre-processing 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9245d1-openshift-default Scaled up replica set e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 from 2 to 4 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Container created 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Container started 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Container created 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Container started 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Add eth0 [10.134.0.152/23] from ovn-kubernetes 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Add eth0 [10.134.0.151/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Readiness probe error: Get "http://10.133.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning Unhealthy pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Readiness probe failed: Get "http://10.133.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Startup probe error: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Startup probe error: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Startup probe failed: Get "http://10.134.0.151:15021/healthz/ready": dial tcp 10.134.0.151:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Startup probe failed: Get "http://10.134.0.152:15021/healthz/ready": dial tcp 10.134.0.152:15021: connect: connection refused 34m Normal Created pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Container created 34m Normal Started pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Container started 34m Normal Pulled pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-d30e9f-5fbc676779-sxx2g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-d30e9f-5f9c9d8546-b4z82 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning ProbeError pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Readiness probe error: Get "http://10.133.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning Unhealthy pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Readiness probe failed: Get "http://10.133.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal SuccessfulCreate replicaset/e2e-inf-b-100ffd-openshift-default-5746c55455 Created pod: e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw 34m Normal ScalingReplicaSet deployment/e2e-inf-b-100ffd-openshift-default Scaled up replica set e2e-inf-b-100ffd-openshift-default-5746c55455 from 0 to 1 34m Normal AddedLabel gateway/e2e-inf-b-100ffd Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-100ffd 34m Normal Pulled pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Container created 34m Normal Started pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Container started 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default New size: 8; reason: 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9245d1-openshift-default Scaled up replica set e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 from 4 to 8 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-e7f767-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-e7f767-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Container created 34m Warning Unhealthy pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Startup probe failed: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Startup probe error: Get "http://10.133.0.210:15021/healthz/ready": dial tcp 10.133.0.210:15021: connect: connection refused... 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Add eth0 [10.132.0.126/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Add eth0 [10.134.0.153/23] from ovn-kubernetes 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Container started 34m Normal AddedInterface pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Add eth0 [10.133.0.220/23] from ovn-kubernetes 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Add eth0 [10.133.0.221/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Container created 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Add eth0 [10.133.0.222/23] from ovn-kubernetes 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Container started 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Container created 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Container started 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Container created 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Container started 34m Warning ProbeError pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-100ffd Scaled up replica set payload-pre-processing-e2e-inf-b-100ffd-7499877555 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-100ffd-6454564456 Error creating: pods "payload-processing-e2e-inf-b-100ffd-6454564456-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-100ffd: serviceaccount "payload-processing-e2e-inf-b-100ffd" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-100ffd Scaled up replica set payload-processing-e2e-inf-b-100ffd-6454564456 from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-100ffd-7499877555 Error creating: pods "payload-pre-processing-e2e-inf-b-100ffd-7499877555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-100ffd: serviceaccount "payload-processing-e2e-inf-b-100ffd" not found 34m Warning ProbeError pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Startup probe error: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-100ffd-6454564456 Created pod: payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-100ffd-7499877555 Created pod: payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Startup probe failed: Get "http://10.134.0.153:15021/healthz/ready": dial tcp 10.134.0.153:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal Created pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Container created 34m Normal Killing pod/e2e-api-a-67440d-openshift-default-5584bd5bc-t5m4b Stopping container istio-proxy 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-537001-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Started pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Container started 34m Normal Pulled pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Add eth0 [10.132.0.127/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-537001-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Startup probe failed: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused 34m Normal Pulled pod/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k Container created 34m Normal AddedInterface pod/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k Add eth0 [10.133.0.223/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 34m Normal Started pod/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k Container started 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Startup probe error: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused... 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9245d1-openshift-default New size: 10; reason: 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Add eth0 [10.132.0.128/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 Created pod: e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Container started 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Container created 34m Normal Pulled pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Container started 34m Normal Created pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Container created 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9245d1-openshift-default Scaled up replica set e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7 from 8 to 10 34m Normal AddedInterface pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Add eth0 [10.133.0.225/23] from ovn-kubernetes 34m Normal Killing pod/payload-processing-e2e-api-b-734d43-f88dd95f4-tsstp Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-api-b-734d43-67ff74f87f-fzb8k Stopping container payload-pre-processing 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Startup probe failed: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Startup probe error: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Startup probe error: Get "http://10.133.0.225:15021/healthz/ready": dial tcp 10.133.0.225:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Startup probe failed: Get "http://10.132.0.128:15021/healthz/ready": dial tcp 10.132.0.128:15021: connect: connection refused 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-537001-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-537001-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-e7f767-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-e7f767-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv 34m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z 34m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled down replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 7 to 6 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 6; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-sdn9z Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 3; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default Scaled down replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 4 to 3 34m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default New size: 3; reason: All metrics below target 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd 34m Normal Killing pod/e2e-inf-a-d30e9f-openshift-default-776cc5cb65-lxzj4 Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-58tcd Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-9wzsv Readiness probe error: HTTP probe failed with statuscode: 503... 33m Normal Started pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Container started 33m Normal Created pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Container created 33m Normal Pulled pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning Unhealthy pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4f415d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4f415d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Stopping container payload-pre-processing 33m Normal AddedLabel gateway/e2e-inf-b-2305f4 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-2305f4 33m Normal SuccessfulCreate replicaset/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf Created pod: e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk 33m Normal ScalingReplicaSet deployment/e2e-inf-b-2305f4-openshift-default Scaled up replica set e2e-inf-b-2305f4-openshift-default-7f9d57bfdf from 0 to 1 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 10 to 8 33m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-6h2wl Stopping container istio-proxy 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4f415d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal AddedInterface pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Add eth0 [10.133.0.228/23] from ovn-kubernetes 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4f415d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-kpx2s 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-6h2wl 33m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-kpx2s Stopping container istio-proxy 33m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/payload-processing-e2e-inf-b-cea203-f4cc9fbcd-bx8rb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc Error creating: pods "payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2305f4: serviceaccount "payload-processing-e2e-inf-b-2305f4" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc Created pod: payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-2305f4 Scaled up replica set payload-pre-processing-e2e-inf-b-2305f4-8478986cbc from 0 to 1 33m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-2305f4-cb5597554 Error creating: pods "payload-processing-e2e-inf-b-2305f4-cb5597554-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2305f4: serviceaccount "payload-processing-e2e-inf-b-2305f4" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-2305f4-cb5597554 Created pod: payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-2305f4 Scaled up replica set payload-processing-e2e-inf-b-2305f4-cb5597554 from 0 to 1 33m Normal Started pod/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n Container started 33m Normal Started pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Container started 33m Normal Pulled pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Created pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Container created 33m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Add eth0 [10.132.0.131/23] from ovn-kubernetes 33m Normal Created pod/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n Container created 33m Normal Pulled pod/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n Add eth0 [10.132.0.130/23] from ovn-kubernetes 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-cea203-5779bf5c55-d82pm Readiness probe failed: timeout: failed to connect service "10.133.0.178:50052" within 1s: context deadline exceeded 33m Normal Killing pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-t6zfn Stopping container istio-proxy 33m Normal Pulled pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Container created 33m Normal Started pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Container started 33m Warning ProbeError pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-b-cea203-openshift-default-68cc5c94f7-gwvtn Readiness probe failed: HTTP probe failed with statuscode: 503 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c1fce3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669 Created pod: e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf 33m Normal ScalingReplicaSet deployment/e2e-inf-a-c1fce3-openshift-default Scaled up replica set e2e-inf-a-c1fce3-openshift-default-7f6c4d6669 from 1 to 2 33m Normal AddedInterface pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Add eth0 [10.132.0.132/23] from ovn-kubernetes 33m Warning ProbeError pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Startup probe error: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Startup probe failed: Get "http://10.133.0.217:15021/healthz/ready": dial tcp 10.133.0.217:15021: connect: connection refused 33m Normal Pulled pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Container created 33m Normal Started pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Container started 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-537001-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-inf-b-537001-openshift-default-74f94b549 Created pod: e2e-inf-b-537001-openshift-default-74f94b549-8ws6l 33m Normal AddedInterface pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Add eth0 [10.132.0.133/23] from ovn-kubernetes 33m Normal ScalingReplicaSet deployment/e2e-inf-b-537001-openshift-default Scaled up replica set e2e-inf-b-537001-openshift-default-74f94b549 from 1 to 2 33m Normal Pulled pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Container started 33m Normal Created pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Container created 33m Warning ProbeError pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 32m Normal Pulled pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f21058-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f21058-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Started pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Container started 32m Normal Created pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Container created 32m Normal Killing pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Stopping container istio-proxy 32m Normal AddedLabel gateway/e2e-inf-a-b0b40d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-b0b40d 32m Normal ScalingReplicaSet deployment/e2e-inf-a-b0b40d-openshift-default Scaled up replica set e2e-inf-a-b0b40d-openshift-default-7445b96d99 from 0 to 1 32m Warning Unhealthy pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-api-b-734d43-openshift-default-65fd65866f-lmq6b Readiness probe error: HTTP probe failed with statuscode: 503... 32m Normal SuccessfulCreate replicaset/e2e-inf-a-b0b40d-openshift-default-7445b96d99 Created pod: e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz 32m Normal AddedInterface pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Add eth0 [10.133.0.231/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Startup probe failed: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Startup probe error: Get "http://10.133.0.220:15021/healthz/ready": dial tcp 10.133.0.220:15021: connect: connection refused... 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-b0b40d-858d44967 Created pod: payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb 32m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-b0b40d-6c7b76b864 Error creating: pods "payload-processing-e2e-inf-a-b0b40d-6c7b76b864-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b0b40d: serviceaccount "payload-processing-e2e-inf-a-b0b40d" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-b0b40d-6c7b76b864 Created pod: payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-b0b40d Scaled up replica set payload-processing-e2e-inf-a-b0b40d-6c7b76b864 from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-b0b40d-858d44967 Error creating: pods "payload-pre-processing-e2e-inf-a-b0b40d-858d44967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-b0b40d: serviceaccount "payload-processing-e2e-inf-a-b0b40d" not found 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-b0b40d Scaled up replica set payload-pre-processing-e2e-inf-a-b0b40d-858d44967 from 0 to 1 32m Normal Created pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Container created 32m Normal Pulled pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Add eth0 [10.133.0.234/23] from ovn-kubernetes 32m Normal Started pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Container started 32m Normal Created pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Container created 32m Normal Pulled pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Add eth0 [10.133.0.233/23] from ovn-kubernetes 32m Normal Started pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Container started 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-100ffd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-100ffd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f21058-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f21058-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal ScalingReplicaSet deployment/e2e-inf-b-4f415d-openshift-default Scaled up replica set e2e-inf-b-4f415d-openshift-default-549bd7f477 from 1 to 2 32m Normal SuccessfulCreate replicaset/e2e-inf-b-4f415d-openshift-default-549bd7f477 Created pod: e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb 32m Normal Pulled pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-4f415d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal AddedInterface pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Add eth0 [10.133.0.237/23] from ovn-kubernetes 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-360818-openshift-default Scaled up replica set e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f from 2 to 3 32m Normal Created pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Container created 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c 32m Normal Started pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Container started 32m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Container created 32m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Container started 32m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Add eth0 [10.134.0.160/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Startup probe failed: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused 32m Warning ProbeError pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Startup probe error: Get "http://10.133.0.237:15021/healthz/ready": dial tcp 10.133.0.237:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-100ffd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-100ffd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Startup probe failed: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Startup probe error: Get "http://10.134.0.160:15021/healthz/ready": dial tcp 10.134.0.160:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Readiness probe error: Get "http://10.133.0.228:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 31m Warning Unhealthy pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Readiness probe failed: Get "http://10.133.0.228:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 31m Normal Created pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Container created 31m Normal Pulled pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Container started 31m Warning Unhealthy pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Startup probe failed: Get "http://10.133.0.228:15021/healthz/ready": dial tcp 10.133.0.228:15021: connect: connection refused 31m Warning ProbeError pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Startup probe error: Get "http://10.133.0.228:15021/healthz/ready": dial tcp 10.133.0.228:15021: connect: connection refused... 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2305f4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2305f4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal AddedLabel gateway/e2e-inf-b-4cf8f5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4cf8f5 31m Normal SuccessfulCreate replicaset/e2e-inf-b-4cf8f5-openshift-default-867459b9 Created pod: e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m 31m Normal ScalingReplicaSet deployment/e2e-inf-b-4cf8f5-openshift-default Scaled up replica set e2e-inf-b-4cf8f5-openshift-default-867459b9 from 0 to 1 31m Normal AddedInterface pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Add eth0 [10.133.0.244/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-4cf8f5-68468b9994 Created pod: payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 31m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-4cf8f5-68468b9994 Error creating: pods "payload-processing-e2e-inf-b-4cf8f5-68468b9994-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4cf8f5: serviceaccount "payload-processing-e2e-inf-b-4cf8f5" not found 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-4cf8f5 Scaled up replica set payload-processing-e2e-inf-b-4cf8f5-68468b9994 from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87 Created pod: payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-4cf8f5 Scaled up replica set payload-pre-processing-e2e-inf-b-4cf8f5-685558f87 from 0 to 1 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87 Error creating: pods "payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4cf8f5: serviceaccount "payload-processing-e2e-inf-b-4cf8f5" not found 31m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Add eth0 [10.132.0.143/23] from ovn-kubernetes 31m Normal AddedInterface pod/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 Add eth0 [10.132.0.142/23] from ovn-kubernetes 31m Normal Created pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Container created 31m Normal Started pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Container started 31m Normal Pulled pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Started pod/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 Container started 31m Normal Created pod/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 Container created 31m Normal Pulled pod/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 8 to 6 31m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-ngtcs 31m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-kdl79 31m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-ngtcs Stopping container istio-proxy 31m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-kdl79 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 31m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx 31m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 4; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default Scaled down replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 6 to 4 31m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-810119-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w2-810119-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-810119-openshift-default-869dbf496f from 3 to 2 31m Normal SuccessfulDelete replicaset/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 Deleted pod: e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default New size: 3; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-aeff1d-openshift-default New size: 2; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-worker-w0-aeff1d-openshift-default Scaled down replica set e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867 from 4 to 2 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default New size: 3; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 31m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-vg7ht Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Stopping container istio-proxy 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f Deleted pod: e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 31m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-d9f3ac-openshift-default New size: 2; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w2-810119-openshift-default-869dbf496f Deleted pod: e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs 31m Normal ScalingReplicaSet deployment/e2e-worker-w3-d9f3ac-openshift-default Scaled down replica set e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f from 4 to 2 31m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Stopping container istio-proxy 31m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-66zxl Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zdp9c Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning ProbeError pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2p5tx Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-lmhb5 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-zjww9 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-worker-w2-810119-openshift-default-869dbf496f-qkkjs Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-g5l89 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Readiness probe error: Get "http://10.134.0.110:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 30m Warning Unhealthy pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-2vkg4 Readiness probe failed: Get "http://10.134.0.110:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 30m Normal Started pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Container started 30m Normal Created pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Container created 30m Normal Pulled pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b0b40d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b0b40d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-32bad3-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 30m Normal SuccessfulDelete replicaset/e2e-worker-w6-32bad3-openshift-default-6cc8655c5 Deleted pod: e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm 30m Normal ScalingReplicaSet deployment/e2e-worker-w6-32bad3-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-32bad3-openshift-default-6cc8655c5 from 3 to 2 30m Warning Unhealthy pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Startup probe failed: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Startup probe error: Get "http://10.133.0.231:15021/healthz/ready": dial tcp 10.133.0.231:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Readiness probe failed: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused 30m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-5btfm Readiness probe error: Get "http://10.134.0.96:15021/healthz/ready": dial tcp 10.134.0.96:15021: connect: connection refused... 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f21058-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal SuccessfulCreate replicaset/e2e-inf-a-f21058-openshift-default-57fbbd47dc Created pod: e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg 30m Normal ScalingReplicaSet deployment/e2e-inf-a-f21058-openshift-default Scaled up replica set e2e-inf-a-f21058-openshift-default-57fbbd47dc from 1 to 2 30m Normal Started pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Container started 30m Normal Pulled pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Add eth0 [10.134.0.177/23] from ovn-kubernetes 30m Normal Created pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Container created 30m Warning Unhealthy pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Startup probe failed: Get "http://10.134.0.177:15021/healthz/ready": dial tcp 10.134.0.177:15021: connect: connection refused 30m Warning ProbeError pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Startup probe error: Get "http://10.134.0.177:15021/healthz/ready": dial tcp 10.134.0.177:15021: connect: connection refused... 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-b0b40d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-b0b40d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal Pulled pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Container created 29m Normal Started pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Container started 29m Warning ProbeError pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Startup probe error: Get "http://10.133.0.244:15021/healthz/ready": dial tcp 10.133.0.244:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Startup probe failed: Get "http://10.133.0.244:15021/healthz/ready": dial tcp 10.133.0.244:15021: connect: connection refused 29m Normal Killing pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Stopping container payload-pre-processing 29m Normal Killing pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Stopping container payload-processing 29m Normal Pulled pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Container started 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-c1fce3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal AddedInterface pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Add eth0 [10.134.0.184/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-inf-a-c1fce3-openshift-default Scaled up replica set e2e-inf-a-c1fce3-openshift-default-7f6c4d6669 from 2 to 3 29m Normal SuccessfulCreate replicaset/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669 Created pod: e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 29m Normal Created pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Container created 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4cf8f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4cf8f5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-f7d1a1-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 from 3 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-f7d1a1-openshift-default New size: 2; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9 Deleted pod: e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz 29m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Stopping container istio-proxy 29m Warning ProbeError pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Startup probe error: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Startup probe failed: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused 29m Warning Unhealthy pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-p78pz Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-zprng Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 6 to 5 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-zprng 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4cf8f5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4cf8f5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default New size: 3; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-g6dmb 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-c1fce3-7cc6d59d47-c67hn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-kzgpm Stopping container istio-proxy 29m Warning Unhealthy pod/payload-processing-e2e-inf-a-c1fce3-bcbf85d6b-w92w7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Killing pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Stopping container istio-proxy 29m Warning ProbeError pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-ds4mf Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-inf-a-c1fce3-openshift-default-7f6c4d6669-v9lc9 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Killing pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Stopping container payload-processing 29m Warning ProbeError pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Readiness probe error: Get "http://10.133.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 29m Warning Unhealthy pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Readiness probe failed: Get "http://10.133.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 29m Normal Killing pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Stopping container payload-processing 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-360818-openshift-default Scaled up replica set e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f from 3 to 5 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 28m Normal Killing pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Stopping container payload-pre-processing 28m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Add eth0 [10.134.0.192/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Container started 28m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Container created 28m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Add eth0 [10.133.0.246/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Container started 28m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Container created 28m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning Unhealthy pod/payload-processing-e2e-inf-b-4f415d-5d8f4cb6b-wsmw7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-4f415d-7bb64d8799-rpwsv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal ScalingReplicaSet deployment/e2e-inf-b-100ffd-openshift-default Scaled up replica set e2e-inf-b-100ffd-openshift-default-5746c55455 from 1 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-100ffd-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-inf-b-100ffd-openshift-default-5746c55455 Created pod: e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 28m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 28m Normal AddedInterface pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Add eth0 [10.134.0.195/23] from ovn-kubernetes 28m Normal Created pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Container created 28m Normal Started pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Container started 28m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Startup probe error: Get "http://10.133.0.246:15021/healthz/ready": dial tcp 10.133.0.246:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Startup probe failed: Get "http://10.133.0.246:15021/healthz/ready": dial tcp 10.133.0.246:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 28m Normal Pulled pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Add eth0 [10.132.0.145/23] from ovn-kubernetes 28m Normal Started pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Container started 28m Normal Created pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Container created 28m Normal Pulled pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal SuccessfulCreate replicaset/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf Created pod: e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh 28m Normal ScalingReplicaSet deployment/e2e-inf-b-2305f4-openshift-default Scaled up replica set e2e-inf-b-2305f4-openshift-default-7f9d57bfdf from 1 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-2305f4-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Warning Unhealthy pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Startup probe error: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Startup probe failed: Get "http://10.132.0.145:15021/healthz/ready": dial tcp 10.132.0.145:15021: connect: connection refused 28m Normal Killing pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-2fvj7 Stopping container istio-proxy 28m Normal Killing pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Stopping container istio-proxy 28m Warning ProbeError pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-07e5c2-openshift-default Scaled up replica set e2e-ipp-a-07e5c2-openshift-default-7789c64df5 from 0 to 1 28m Warning Unhealthy pod/payload-processing-e2e-inf-a-16df10-58fb5c86cb-5bhpr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-07e5c2-openshift-default-7789c64df5 Created pod: e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 28m Normal AddedLabel gateway/e2e-ipp-a-07e5c2 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-07e5c2 28m Warning Unhealthy pod/e2e-inf-b-4f415d-openshift-default-549bd7f477-7z2qb Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-16df10-859bf4d79-25wlb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 28m Normal AddedInterface pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Add eth0 [10.133.0.247/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-07e5c2 Scaled up replica set payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99 from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99 Error creating: pods "payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-07e5c2: serviceaccount "payload-processing-e2e-ipp-a-07e5c2" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99 Created pod: payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-07e5c2-f7846c9b Error creating: pods "payload-processing-e2e-ipp-a-07e5c2-f7846c9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-07e5c2: serviceaccount "payload-processing-e2e-ipp-a-07e5c2" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-07e5c2-f7846c9b Created pod: payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-07e5c2 Scaled up replica set payload-processing-e2e-ipp-a-07e5c2-f7846c9b from 0 to 1 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v Container created 28m Normal Created pod/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v Container started 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr Add eth0 [10.133.0.249/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v Add eth0 [10.133.0.248/23] from ovn-kubernetes 28m Normal Killing pod/e2e-inf-a-16df10-openshift-default-6d7dffd555-4rf8r Stopping container istio-proxy 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 Created pod: e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-9d3552-openshift-default Scaled up replica set e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 from 1 to 2 28m Normal Started pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Container started 28m Normal Created pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Container created 28m Normal Pulled pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Add eth0 [10.133.0.250/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/e2e-inf-b-4cf8f5-openshift-default Scaled up replica set e2e-inf-b-4cf8f5-openshift-default-867459b9 from 1 to 2 28m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 5 to 4 28m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 28m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-4cf8f5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-mkrp4 28m Normal SuccessfulCreate replicaset/e2e-inf-b-4cf8f5-openshift-default-867459b9 Created pod: e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 28m Normal Pulled pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Created pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Container created 28m Normal Started pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Container started 28m Normal AddedInterface pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Add eth0 [10.134.0.201/23] from ovn-kubernetes 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-7d1788-openshift-default-86d57f97ff Deleted pod: e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm 28m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Stopping container istio-proxy 28m Normal Killing pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Stopping container payload-processing 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-7d1788-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-7d1788-openshift-default-86d57f97ff from 3 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-7d1788-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal Killing pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Stopping container payload-pre-processing 28m Warning Unhealthy pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Startup probe failed: Get "http://10.133.0.250:15021/healthz/ready": dial tcp 10.133.0.250:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Startup probe error: Get "http://10.133.0.250:15021/healthz/ready": dial tcp 10.133.0.250:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-t78nm Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning ProbeError pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 27m Warning Unhealthy pod/payload-processing-e2e-inf-b-537001-66fd974b46-cbngf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-537001-554c9c6887-rnj4c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/payload-processing-e2e-inf-a-f21058-86d9b6f68-97sdp Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Stopping container payload-pre-processing 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-07e5c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-07e5c2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal Killing pod/payload-processing-e2e-inf-a-e7f767-844d6b5bdd-zlzl2 Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Stopping container payload-pre-processing 27m Normal Killing pod/e2e-inf-b-537001-openshift-default-74f94b549-kh7mf Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Stopping container istio-proxy 27m Normal AddedLabel gateway/e2e-ipp-a-086aee Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-086aee 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-f21058-68466cc45f-d8kgh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-086aee-openshift-default Scaled up replica set e2e-ipp-a-086aee-openshift-default-87d5ddd45 from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-086aee-openshift-default-87d5ddd45 Created pod: e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 27m Normal AddedInterface pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Add eth0 [10.133.0.255/23] from ovn-kubernetes 27m Warning Unhealthy pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Readiness probe failed: HTTP probe failed with statuscode: 503 27m Warning ProbeError pod/e2e-inf-b-537001-openshift-default-74f94b549-8ws6l Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759 Error creating: pods "payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-086aee: serviceaccount "payload-processing-e2e-ipp-a-086aee" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-086aee Scaled up replica set payload-processing-e2e-ipp-a-086aee-79f65bf7f8 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-086aee-79f65bf7f8 Error creating: pods "payload-processing-e2e-ipp-a-086aee-79f65bf7f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-086aee: serviceaccount "payload-processing-e2e-ipp-a-086aee" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-086aee Scaled up replica set payload-pre-processing-e2e-ipp-a-086aee-57bbf65759 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759 Created pod: payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb 27m Normal Killing pod/e2e-inf-a-e7f767-openshift-default-7d54d7f747-82h7m Stopping container istio-proxy 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb Add eth0 [10.133.1.0/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-086aee-79f65bf7f8 Created pod: payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 Container created 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb Container created 27m Normal Started pod/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 Container started 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 Add eth0 [10.133.1.1/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-e7f767-7c4dc78559-26s6h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Killing pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-gfjpg Stopping container istio-proxy 27m Normal Killing pod/e2e-inf-a-f21058-openshift-default-57fbbd47dc-rlbcg Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-b-100ffd-6454564456-7jm7k Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-b-2305f4-cb5597554-rbk5n Stopping container payload-processing 26m Normal Pulled pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-100ffd-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-07e5c2-554566bb99-vfv6v Stopping container payload-pre-processing 26m Normal ScalingReplicaSet deployment/e2e-inf-b-100ffd-openshift-default Scaled up replica set e2e-inf-b-100ffd-openshift-default-5746c55455 from 2 to 3 26m Normal SuccessfulCreate replicaset/e2e-inf-b-100ffd-openshift-default-5746c55455 Created pod: e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b 26m Normal Killing pod/payload-processing-e2e-ipp-a-07e5c2-f7846c9b-jbxrr Stopping container payload-processing 26m Normal AddedInterface pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Add eth0 [10.132.0.146/23] from ovn-kubernetes 26m Normal Created pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Container created 26m Normal Started pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Container started 26m Warning ProbeError pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Readiness probe error: Get "http://10.133.0.247:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Readiness probe failed: Get "http://10.133.0.247:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Normal Created pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Container created 26m Normal Pulled pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Container started 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-100ffd-7499877555-ngfvx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning ProbeError pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Startup probe error: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Startup probe failed: Get "http://10.132.0.146:15021/healthz/ready": dial tcp 10.132.0.146:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Startup probe error: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Startup probe failed: Get "http://10.133.0.247:15021/healthz/ready": dial tcp 10.133.0.247:15021: connect: connection refused 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-2305f4-8478986cbc-p5nx8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Normal Killing pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Stopping container istio-proxy 26m Warning ProbeError pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning Unhealthy pod/e2e-inf-b-100ffd-openshift-default-5746c55455-t4v7b Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Readiness probe error: HTTP probe failed with statuscode: 503... 26m Warning Unhealthy pod/e2e-inf-b-100ffd-openshift-default-5746c55455-tdz84 Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-inf-b-100ffd-openshift-default-5746c55455-lngvw Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal AddedLabel gateway/e2e-ipp-a-146bb7 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-146bb7 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-146bb7-openshift-default Scaled up replica set e2e-ipp-a-146bb7-openshift-default-79578d4855 from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-146bb7-openshift-default-79578d4855 Created pod: e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h 26m Normal AddedInterface pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Add eth0 [10.133.1.4/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67 Error creating: pods "payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-07e5c2: serviceaccount "payload-processing-e2e-ipp-a-07e5c2" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78 Created pod: payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-07e5c2 Scaled up replica set payload-processing-e2e-ipp-a-07e5c2-86ddf96f78 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78 Error creating: pods "payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-07e5c2: serviceaccount "payload-processing-e2e-ipp-a-07e5c2" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67 Created pod: payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-07e5c2 Scaled up replica set payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-146bb7 Scaled up replica set payload-pre-processing-e2e-ipp-a-146bb7-68c884f987 from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987 Error creating: pods "payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-146bb7: serviceaccount "payload-processing-e2e-ipp-a-146bb7" not found 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Add eth0 [10.133.1.5/23] from ovn-kubernetes 26m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987 Created pod: payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d Error creating: pods "payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-146bb7: serviceaccount "payload-processing-e2e-ipp-a-146bb7" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d Created pod: payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Add eth0 [10.133.1.6/23] from ovn-kubernetes 26m Normal Pulling pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-146bb7 Scaled up replica set payload-processing-e2e-ipp-a-146bb7-7cdf4b996d from 0 to 1 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 Container started 26m Normal Started pod/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn Add eth0 [10.133.1.8/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 Add eth0 [10.133.1.7/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn Container created 26m Normal Killing pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-t84tk Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-2305f4-openshift-default-7f9d57bfdf-xvwhh Stopping container istio-proxy 26m Normal Pulled pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.987s (2.987s including waiting). Image size: 166612641 bytes. 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Container created 26m Normal AddedLabel gateway/e2e-shared-a-w0-d823d1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-d823d1 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 3.208s (3.208s including waiting). Image size: 166612641 bytes. 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Container started 26m Normal Started pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Container started 26m Normal Created pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Container created 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479 Created pod: e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-d823d1-openshift-default Scaled up replica set e2e-shared-a-w0-d823d1-openshift-default-566cfb7479 from 0 to 1 26m Normal AddedInterface pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Add eth0 [10.133.1.9/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-d823d1 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48 Error creating: pods "payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-d823d1: serviceaccount "payload-processing-e2e-shared-a-w0-d823d1" not found 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55 Error creating: pods "payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-d823d1: serviceaccount "payload-processing-e2e-shared-a-w0-d823d1" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55 Created pod: payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-d823d1 Scaled up replica set payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48 Created pod: payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf 26m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Add eth0 [10.133.1.10/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Add eth0 [10.133.1.11/23] from ovn-kubernetes 26m Normal Started pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Container started 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-07e5c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Pulled pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Container created 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-07e5c2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Container started 26m Normal Created pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Container created 26m Normal AddedLabel gateway/e2e-ipp-b-734274 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-734274 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-734274-openshift-default Scaled up replica set e2e-ipp-b-734274-openshift-default-5ddb7bbcbc from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc Created pod: e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx 26m Normal Killing pod/payload-processing-e2e-ipp-a-086aee-79f65bf7f8-cbnz8 Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-086aee-57bbf65759-dzpzb Stopping container payload-pre-processing 26m Normal AddedInterface pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Add eth0 [10.133.1.12/23] from ovn-kubernetes 26m Normal Started pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Container started 26m Normal Created pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Container created 26m Normal Pulled pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-734274 Scaled up replica set payload-pre-processing-e2e-ipp-b-734274-648f7f965f from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-734274-648f7f965f Error creating: pods "payload-pre-processing-e2e-ipp-b-734274-648f7f965f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-734274: serviceaccount "payload-processing-e2e-ipp-b-734274" not found 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-734274-659ddbf7cb Error creating: pods "payload-processing-e2e-ipp-b-734274-659ddbf7cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-734274: serviceaccount "payload-processing-e2e-ipp-b-734274" not found 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-734274 Scaled up replica set payload-processing-e2e-ipp-b-734274-659ddbf7cb from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-734274-648f7f965f Created pod: payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 Add eth0 [10.132.0.147/23] from ovn-kubernetes 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-734274-659ddbf7cb Created pod: payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 Add eth0 [10.132.0.148/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 Container started 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 Created pod: e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-9d3552-openshift-default Scaled up replica set e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 from 2 to 3 25m Warning ProbeError pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Startup probe error: Get "http://10.133.0.255:15021/healthz/ready": dial tcp 10.133.0.255:15021: connect: connection refused... 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Warning Unhealthy pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Startup probe failed: Get "http://10.133.0.255:15021/healthz/ready": dial tcp 10.133.0.255:15021: connect: connection refused 25m Normal AddedInterface pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Add eth0 [10.132.0.149/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Container created 25m Normal Started pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Container started 25m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-6qcgn 25m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-6qcgn Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 3 25m Warning Unhealthy pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 25m Normal ScalingReplicaSet deployment/e2e-worker-w1-e11f03-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 from 3 to 2 25m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl Stopping container istio-proxy 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-e11f03-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-360818-openshift-default Scaled up replica set e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f from 5 to 6 25m Normal SuccessfulDelete replicaset/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7 Deleted pod: e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-9vmrl 25m Warning ProbeError pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Add eth0 [10.132.0.150/23] from ovn-kubernetes 25m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Container started 25m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Container created 25m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Killing pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Stopping container payload-processing 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-086aee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-086aee-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Startup probe error: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Startup probe failed: Get "http://10.132.0.150:15021/healthz/ready": dial tcp 10.132.0.150:15021: connect: connection refused 25m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 25m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 25m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-5fnmn Stopping container discovery 25m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-5fnmn 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-086aee-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-086aee-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-086aee Scaled up replica set payload-processing-e2e-ipp-a-086aee-78d559cdfb from 0 to 1 25m Warning Unhealthy pod/payload-processing-e2e-inf-a-b0b40d-6c7b76b864-4lrtx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-b0b40d-858d44967-gdzsb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c Error creating: pods "payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-086aee: serviceaccount "payload-processing-e2e-ipp-a-086aee" not found 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-086aee-78d559cdfb Error creating: pods "payload-processing-e2e-ipp-a-086aee-78d559cdfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-086aee: serviceaccount "payload-processing-e2e-ipp-a-086aee" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-086aee-78d559cdfb Created pod: payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-086aee Scaled up replica set payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c Created pod: payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp 25m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Add eth0 [10.132.0.153/23] from ovn-kubernetes 25m Normal Pulling pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Add eth0 [10.132.0.152/23] from ovn-kubernetes 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.833s (1.833s including waiting). Image size: 166612641 bytes. 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Container started 25m Normal Pulled pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.795s (1.795s including waiting). Image size: 166612641 bytes. 25m Normal Started pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Container created 25m Normal Created pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Container created 25m Normal Started pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Container started 25m Normal Created pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Container created 25m Normal Pulled pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-146bb7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-146bb7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-6b5f19-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 25m Warning Unhealthy pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Startup probe failed: Get "http://10.133.1.4:15021/healthz/ready": dial tcp 10.133.1.4:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Startup probe error: Get "http://10.133.1.4:15021/healthz/ready": dial tcp 10.133.1.4:15021: connect: connection refused... 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-6b5f19-openshift-default Scaled up replica set e2e-shared-a-w5-6b5f19-openshift-default-85665454c from 1 to 2 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-6b5f19-openshift-default-85665454c Created pod: e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh 25m Normal Started pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Container started 25m Normal Created pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Container created 25m Normal Pulled pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Add eth0 [10.132.0.155/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757 Created pod: e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs 25m Normal ScalingReplicaSet deployment/e2e-ipp-b-1a0f7e-openshift-default Scaled up replica set e2e-ipp-b-1a0f7e-openshift-default-5d9c577757 from 0 to 1 25m Normal AddedLabel gateway/e2e-ipp-b-1a0f7e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-1a0f7e 25m Normal AddedInterface pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Add eth0 [10.133.1.14/23] from ovn-kubernetes 25m Warning Unhealthy pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Startup probe failed: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Startup probe error: Get "http://10.132.0.155:15021/healthz/ready": dial tcp 10.132.0.155:15021: connect: connection refused... 25m Normal Killing pod/e2e-inf-a-b0b40d-openshift-default-7445b96d99-rjmcz Stopping container istio-proxy 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc Error creating: pods "payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a0f7e: serviceaccount "payload-processing-e2e-ipp-b-1a0f7e" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc Created pod: payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk 24m Normal Killing pod/payload-processing-e2e-ipp-a-146bb7-7cdf4b996d-ntfgn Stopping container payload-processing 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8 Error creating: pods "payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a0f7e: serviceaccount "payload-processing-e2e-ipp-b-1a0f7e" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8 Created pod: payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-146bb7-68c884f987-fsmc2 Stopping container payload-pre-processing 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1a0f7e Scaled up replica set payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8 from 0 to 1 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q Add eth0 [10.133.1.15/23] from ovn-kubernetes 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk Add eth0 [10.132.0.156/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1a0f7e Scaled up replica set payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc from 0 to 1 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q Container started 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q Container created 24m Normal Killing pod/payload-processing-e2e-inf-b-4cf8f5-68468b9994-vb6n4 Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Stopping container payload-pre-processing 24m Normal AddedLabel gateway/e2e-shared-b-w0-874ab5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-874ab5 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-874ab5-openshift-default Scaled up replica set e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5 from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5 Created pod: e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt 24m Normal AddedInterface pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Add eth0 [10.133.1.16/23] from ovn-kubernetes 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-874ab5: serviceaccount "payload-processing-e2e-shared-b-w0-874ab5" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659 Created pod: payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn 24m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b Error creating: pods "payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-874ab5: serviceaccount "payload-processing-e2e-shared-b-w0-874ab5" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-874ab5 Scaled up replica set payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b Created pod: payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-874ab5 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659 from 0 to 1 24m Normal Pulled pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Container started 24m Normal Created pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Container created 24m Normal Started pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Container started 24m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Add eth0 [10.133.1.17/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Container created 24m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Add eth0 [10.133.1.18/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Readiness probe failed: Get "http://10.133.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-4cf8f5-685558f87-f6gbw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-processing-e2e-ipp-b-734274-659ddbf7cb-6qxs5 Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-734274-648f7f965f-z5qm9 Stopping container payload-pre-processing 24m Warning ProbeError pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Readiness probe error: Get "http://10.133.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-146bb7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-146bb7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Started pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Container started 24m Normal Created pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Container created 24m Normal Pulled pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf Error creating: pods "payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-146bb7: serviceaccount "payload-processing-e2e-ipp-a-146bb7" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-146bb7 Scaled up replica set payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-146bb7-798dcd9494 Error creating: pods "payload-processing-e2e-ipp-a-146bb7-798dcd9494-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-146bb7: serviceaccount "payload-processing-e2e-ipp-a-146bb7" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-146bb7 Scaled up replica set payload-processing-e2e-ipp-a-146bb7-798dcd9494 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-146bb7-798dcd9494 Created pod: payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 24m Normal Pulled pod/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 Add eth0 [10.132.0.157/23] from ovn-kubernetes 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 Container started 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf Created pod: payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 24m Warning ProbeError pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Startup probe error: Get "http://10.133.1.9:15021/healthz/ready": dial tcp 10.133.1.9:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Startup probe failed: Get "http://10.133.1.9:15021/healthz/ready": dial tcp 10.133.1.9:15021: connect: connection refused 24m Normal Pulled pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 Add eth0 [10.132.0.158/23] from ovn-kubernetes 24m Normal Created pod/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 Container created 24m Normal Started pod/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 Container started 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-d823d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Created pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Container created 24m Normal Started pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Container started 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-d823d1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-734274-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f Created pod: e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-360818-openshift-default Scaled up replica set e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f from 6 to 8 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-734274-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Add eth0 [10.132.0.159/23] from ovn-kubernetes 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-360818-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 24m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Container started 24m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Container created 24m Normal AddedInterface pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Add eth0 [10.133.1.19/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Container created 24m Normal Started pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Container started 24m Warning Unhealthy pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Startup probe failed: Get "http://10.133.1.12:15021/healthz/ready": dial tcp 10.133.1.12:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Startup probe error: Get "http://10.133.1.12:15021/healthz/ready": dial tcp 10.133.1.12:15021: connect: connection refused... 24m Normal Created pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Container created 24m Normal Pulled pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-ae1dd9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal Started pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Container started 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff Created pod: e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 24m Normal AddedInterface pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Add eth0 [10.132.0.161/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-ae1dd9-openshift-default Scaled up replica set e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff from 1 to 2 24m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Startup probe error: Get "http://10.133.1.19:15021/healthz/ready": dial tcp 10.133.1.19:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Startup probe failed: Get "http://10.133.1.19:15021/healthz/ready": dial tcp 10.133.1.19:15021: connect: connection refused 24m Normal Killing pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-f74c4 Stopping container istio-proxy 24m Normal Killing pod/e2e-inf-b-4cf8f5-openshift-default-867459b9-9ww7m Stopping container istio-proxy 24m Normal AddedLabel gateway/e2e-shared-a-w6-4b5459 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-4b5459 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc Created pod: e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 24m Warning ProbeError pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Startup probe error: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Startup probe failed: Get "http://10.132.0.161:15021/healthz/ready": dial tcp 10.132.0.161:15021: connect: connection refused 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-4b5459-openshift-default Scaled up replica set e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-948965-openshift-default-5555b799b7 Created pod: e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-948965-openshift-default Scaled up replica set e2e-ipp-b-948965-openshift-default-5555b799b7 from 0 to 1 24m Normal AddedLabel gateway/e2e-ipp-b-948965 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-948965 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-d823d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal AddedInterface pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Add eth0 [10.134.0.220/23] from ovn-kubernetes 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-d823d1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Normal AddedInterface pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Add eth0 [10.134.0.219/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-4b5459 Scaled up replica set payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b Created pod: payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f Created pod: payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-4b5459 Scaled up replica set payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b Error creating: pods "payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-4b5459: serviceaccount "payload-processing-e2e-shared-a-w6-4b5459" not found 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f Error creating: pods "payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-4b5459: serviceaccount "payload-processing-e2e-shared-a-w6-4b5459" not found 23m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Container started 23m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Container started 23m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Container created 23m Normal Created pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Container created 23m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Add eth0 [10.132.0.162/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Add eth0 [10.133.1.20/23] from ovn-kubernetes 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-948965 Scaled up replica set payload-processing-e2e-ipp-b-948965-5d4df5d896 from 0 to 1 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc Error creating: pods "payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-948965: serviceaccount "payload-processing-e2e-ipp-b-948965" not found 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-948965 Scaled up replica set payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc Created pod: payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh Add eth0 [10.132.0.163/23] from ovn-kubernetes 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-948965-5d4df5d896 Error creating: pods "payload-processing-e2e-ipp-b-948965-5d4df5d896-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-948965: serviceaccount "payload-processing-e2e-ipp-b-948965" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-948965-5d4df5d896 Created pod: payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh 23m Normal Created pod/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh Container started 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd Container started 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd Container created 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd Add eth0 [10.134.0.221/23] from ovn-kubernetes 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-734274-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-734274-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-734274 Scaled up replica set payload-pre-processing-e2e-ipp-b-734274-758c649cdc from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-734274 Scaled up replica set payload-processing-e2e-ipp-b-734274-84746f954 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-734274-758c649cdc Created pod: payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-734274-758c649cdc Error creating: pods "payload-pre-processing-e2e-ipp-b-734274-758c649cdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-734274: serviceaccount "payload-processing-e2e-ipp-b-734274" not found 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-734274-84746f954 Error creating: pods "payload-processing-e2e-ipp-b-734274-84746f954-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-734274: serviceaccount "payload-processing-e2e-ipp-b-734274" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-734274-84746f954 Created pod: payload-processing-e2e-ipp-b-734274-84746f954-z25tg 23m Normal Started pod/payload-processing-e2e-ipp-b-734274-84746f954-z25tg Container started 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd Container started 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-734274-84746f954-z25tg Add eth0 [10.132.0.164/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-b-734274-84746f954-z25tg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-b-734274-84746f954-z25tg Container created 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd Add eth0 [10.132.0.165/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd Container created 23m Normal Pulled pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Container created 23m Normal Started pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Container started 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Stopping container payload-pre-processing 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-948965-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-948965-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Stopping container payload-processing 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1a0f7e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1a0f7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning ProbeError pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Startup probe error: Get "http://10.133.1.14:15021/healthz/ready": dial tcp 10.133.1.14:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Startup probe failed: Get "http://10.133.1.14:15021/healthz/ready": dial tcp 10.133.1.14:15021: connect: connection refused 23m Normal Killing pod/payload-processing-e2e-ipp-a-07e5c2-86ddf96f78-mndhm Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-07e5c2-64b747b67-zh9zg Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-1a0f7e-5f8d6bf7f8-tsd7q Stopping container payload-processing 23m Normal Killing pod/e2e-ipp-a-07e5c2-openshift-default-7789c64df5-r9956 Stopping container istio-proxy 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1a0f7e-654f749dbc-6lrjk Stopping container payload-pre-processing 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-9245d1-69495f589f-cv5xt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-9245d1-6c69f96f98-x77n7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1a0f7e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1a0f7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Killing pod/payload-processing-e2e-ipp-b-734274-84746f954-z25tg Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-734274-758c649cdc-87sfd Stopping container payload-pre-processing 22m Normal Started pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Container started 22m Normal Pulled pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Container created 22m Normal Killing pod/e2e-ipp-b-734274-openshift-default-5ddb7bbcbc-n6wwx Stopping container istio-proxy 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f Error creating: pods "payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a0f7e: serviceaccount "payload-processing-e2e-ipp-b-1a0f7e" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f Created pod: payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns 22m Warning Unhealthy pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Startup probe failed: Get "http://10.133.1.16:15021/healthz/ready": dial tcp 10.133.1.16:15021: connect: connection refused 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1a0f7e-55577bf768 Error creating: pods "payload-processing-e2e-ipp-b-1a0f7e-55577bf768-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1a0f7e: serviceaccount "payload-processing-e2e-ipp-b-1a0f7e" not found 22m Warning ProbeError pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Startup probe error: Get "http://10.133.1.16:15021/healthz/ready": dial tcp 10.133.1.16:15021: connect: connection refused... 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1a0f7e Scaled up replica set payload-processing-e2e-ipp-b-1a0f7e-55577bf768 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1a0f7e Scaled up replica set payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f from 0 to 1 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-8qsrf Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-qb9ff Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-gwffx Stopping container istio-proxy 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1a0f7e-55577bf768 Created pod: payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 Add eth0 [10.132.0.166/23] from ovn-kubernetes 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns Add eth0 [10.133.1.25/23] from ovn-kubernetes 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-sdrwh Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-wjs4k Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-lfvgl Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-nnbb2 Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-b7c9h Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-p46sq Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-a-w4-9245d1-openshift-default-5b5f444c7-rbz7b Stopping container istio-proxy 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns Container started 22m Normal Pulled pod/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 Container created 22m Normal Started pod/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 Container started 22m Normal Killing pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Stopping container payload-processing 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-874ab5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-874ab5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal Killing pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Stopping container payload-pre-processing 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-948965-584cd8fdbc-x49rd Stopping container payload-pre-processing 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-8b0787-openshift-default Scaled up replica set e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5 from 0 to 1 22m Normal AddedLabel gateway/e2e-shared-b-w6-8b0787 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-8b0787 22m Normal Killing pod/payload-processing-e2e-ipp-b-948965-5d4df5d896-tqrqh Stopping container payload-processing 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5 Created pod: e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 22m Normal AddedInterface pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Add eth0 [10.133.1.26/23] from ovn-kubernetes 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-874ab5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-874ab5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd Created pod: payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 22m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd Error creating: pods "payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-8b0787: serviceaccount "payload-processing-e2e-shared-b-w6-8b0787" not found 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-8b0787: serviceaccount "payload-processing-e2e-shared-b-w6-8b0787" not found 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5 Created pod: payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-8b0787 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-8b0787 Scaled up replica set payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd from 0 to 1 22m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Add eth0 [10.133.1.27/23] from ovn-kubernetes 22m Normal Created pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Container created 22m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Add eth0 [10.133.1.28/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Container started 22m Normal Pulled pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Container started 22m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Container created 22m Warning Unhealthy pod/payload-processing-e2e-worker-w2-810119-7fc6f9f6cd-dnmhs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-810119-7985cbbff5-t7v5n Readiness probe failed: timeout: failed to connect service "10.133.0.60:50052" within 1s: context deadline exceeded 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-dfz2c Stopping container istio-proxy 22m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-69ckk Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Stopping container istio-proxy 22m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Readiness probe error: HTTP probe failed with statuscode: 503... 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-2ljdf Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-h82z8 Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-t96bn Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-rjrks Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Started pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Container started 22m Normal Created pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Container created 22m Normal Pulled pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-gcf5w Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-360818-848768dc78-8jws2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Started pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Container started 22m Normal Created pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Container created 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-360818-55b6c65c9f-2vjtg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Pulled pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedLabel gateway/e2e-ipp-a-e9528f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e9528f 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-4b5459-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-4b5459-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c Created pod: e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-e9528f-openshift-default Scaled up replica set e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c from 0 to 1 22m Normal AddedInterface pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Add eth0 [10.133.1.29/23] from ovn-kubernetes 22m Warning ProbeError pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-shared-b-w4-360818-openshift-default-65c5f7bf4f-4q4vn Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Created pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Container created 22m Normal AddedInterface pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Add eth0 [10.132.0.167/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479 Created pod: e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-d823d1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-d823d1-openshift-default Scaled up replica set e2e-shared-a-w0-d823d1-openshift-default-566cfb7479 from 1 to 2 22m Normal Started pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Container started 22m Normal Pulled pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Warning ProbeError pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Startup probe error: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Startup probe failed: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Startup probe error: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Startup probe failed: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e9528f Scaled up replica set payload-processing-e2e-ipp-a-e9528f-dfff754fd from 0 to 1 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 Add eth0 [10.133.1.31/23] from ovn-kubernetes 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e9528f Scaled up replica set payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e9528f-dfff754fd Created pod: payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn Add eth0 [10.133.1.32/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e9528f-dfff754fd Error creating: pods "payload-processing-e2e-ipp-a-e9528f-dfff754fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e9528f: serviceaccount "payload-processing-e2e-ipp-a-e9528f" not found 22m Normal Killing pod/payload-processing-e2e-worker-w4-7d1788-784cfd57f6-4v4rx Stopping container payload-processing 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f Error creating: pods "payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e9528f: serviceaccount "payload-processing-e2e-ipp-a-e9528f" not found 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 Container created 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f Created pod: payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 22m Normal Pulled pod/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Killing pod/payload-pre-processing-e2e-worker-w4-7d1788-85f96f7f55-bht78 Stopping container payload-pre-processing 21m Normal Started pod/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn Container started 21m Normal Created pod/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn Container created 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 Container started 21m Warning Unhealthy pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-948965 Scaled up replica set payload-pre-processing-e2e-ipp-b-948965-6686d85647 from 0 to 1 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-948965-5dbd868586 Error creating: pods "payload-processing-e2e-ipp-b-948965-5dbd868586-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-948965: serviceaccount "payload-processing-e2e-ipp-b-948965" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-948965 Scaled up replica set payload-processing-e2e-ipp-b-948965-5dbd868586 from 0 to 1 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-948965-6686d85647 Error creating: pods "payload-pre-processing-e2e-ipp-b-948965-6686d85647-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-948965: serviceaccount "payload-processing-e2e-ipp-b-948965" not found 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-948965-6686d85647 Created pod: payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf 21m Normal Created pod/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w Container created 21m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 21m Normal Pulled pod/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-948965-5dbd868586 Created pod: payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w Add eth0 [10.133.1.33/23] from ovn-kubernetes 21m Normal Started pod/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w Container started 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Add eth0 [10.134.0.232/23] from ovn-kubernetes 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Container started 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.185s (1.185s including waiting). Image size: 166612641 bytes. 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Container created 21m Normal Killing pod/payload-processing-e2e-ipp-a-086aee-78d559cdfb-lppwv Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-086aee-78d7546f8c-x7shp Stopping container payload-pre-processing 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-948965-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-948965-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal Killing pod/e2e-ipp-a-086aee-openshift-default-87d5ddd45-k76z8 Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-fkd8q Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w2-810119-openshift-default-869dbf496f-5ml9z Stopping container istio-proxy 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-4b5459-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-4b5459-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-874ab5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 21m Normal Pulled pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Add eth0 [10.134.0.238/23] from ovn-kubernetes 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5 Created pod: e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-874ab5-openshift-default Scaled up replica set e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5 from 1 to 2 21m Normal Created pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Container created 21m Normal Started pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Container started 21m Warning ProbeError pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Startup probe error: Get "http://10.134.0.238:15021/healthz/ready": dial tcp 10.134.0.238:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Startup probe failed: Get "http://10.134.0.238:15021/healthz/ready": dial tcp 10.134.0.238:15021: connect: connection refused 21m Normal Pulled pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Container created 21m Normal Started pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Container started 21m Warning Unhealthy pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Startup probe failed: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Startup probe error: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused... 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-146bb7-746dc644cf-d5952 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-1a0f7e-55577bf768-j2nh5 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1a0f7e-767f74dd9f-d7nns Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-146bb7-798dcd9494-62zq4 Stopping container payload-processing 20m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 Deleted pod: e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 20m Normal Killing pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-chtm9 Stopping container istio-proxy 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-9d3552-openshift-default New size: 2; reason: All metrics below target 20m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-9d3552-openshift-default Scaled down replica set e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7 from 3 to 2 20m Normal Created pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Container created 20m Normal Started pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Container started 20m Normal Pulled pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-sj9pb Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Stopping container istio-proxy 20m Normal Killing pod/e2e-ipp-a-146bb7-openshift-default-79578d4855-d8f8h Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-worker-w4-7d1788-openshift-default-86d57f97ff-q5mfw Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/e2e-ipp-b-1a0f7e-openshift-default-5d9c577757-td9qs Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e9528f-6d94c9845f-dwvb8 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-e9528f-dfff754fd-s8kdn Stopping container payload-processing 20m Warning Unhealthy pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Startup probe failed: Get "http://10.133.1.29:15021/healthz/ready": dial tcp 10.133.1.29:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Startup probe error: Get "http://10.133.1.29:15021/healthz/ready": dial tcp 10.133.1.29:15021: connect: connection refused... 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-8b0787-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-8b0787-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e9528f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e9528f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/payload-processing-e2e-shared-a-w3-723ffc-84c8887d64-nsq4h Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-723ffc-94746cdbc-v9d98 Stopping container payload-pre-processing 20m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc Created pod: e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl 20m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-4b5459-openshift-default Scaled up replica set e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc from 1 to 2 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e9528f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e9528f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-4b5459-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e9528f Scaled up replica set payload-processing-e2e-ipp-a-e9528f-65b447dd7b from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e9528f Scaled up replica set payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7 from 0 to 1 20m Normal Started pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Container started 20m Normal Created pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Container created 20m Normal Pulled pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Add eth0 [10.132.0.170/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b Add eth0 [10.132.0.171/23] from ovn-kubernetes 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7 Error creating: pods "payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e9528f: serviceaccount "payload-processing-e2e-ipp-a-e9528f" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7 Created pod: payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e9528f-65b447dd7b Error creating: pods "payload-processing-e2e-ipp-a-e9528f-65b447dd7b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e9528f: serviceaccount "payload-processing-e2e-ipp-a-e9528f" not found 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e9528f-65b447dd7b Created pod: payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b Container created 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b Container started 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 Add eth0 [10.132.0.172/23] from ovn-kubernetes 20m Normal Started pod/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 Container started 20m Normal Pulled pod/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 Container created 20m Warning Unhealthy pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Startup probe failed: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused 20m Warning ProbeError pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Startup probe error: Get "http://10.132.0.170:15021/healthz/ready": dial tcp 10.132.0.170:15021: connect: connection refused... 20m Normal Killing pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-948965-6686d85647-4kxzf Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-948965-5dbd868586-xww5w Stopping container payload-processing 20m Warning Unhealthy pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-a-w3-723ffc-openshift-default-5b57c844cf-cnsjp Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal AddedLabel gateway/e2e-ipp-b-055daa Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-055daa 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-055daa-openshift-default Scaled up replica set e2e-ipp-b-055daa-openshift-default-68c775fb45 from 0 to 1 20m Normal AddedInterface pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Add eth0 [10.133.1.35/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-055daa-openshift-default-68c775fb45 Created pod: e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 20m Normal Killing pod/e2e-ipp-b-948965-openshift-default-5555b799b7-9wj5r Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-8e2727-747c8cc64c-lhsqt Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w3-8e2727-5dbb46544b-5knb4 Stopping container payload-processing 19m Normal ScalingReplicaSet deployment/e2e-ipp-a-cf605a-openshift-default Scaled up replica set e2e-ipp-a-cf605a-openshift-default-79f69d44f from 0 to 1 19m Normal AddedLabel gateway/e2e-ipp-a-cf605a Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-cf605a 19m Normal AddedInterface pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Add eth0 [10.133.1.36/23] from ovn-kubernetes 19m Normal SuccessfulCreate replicaset/e2e-ipp-a-cf605a-openshift-default-79f69d44f Created pod: e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-055daa Scaled up replica set payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc from 0 to 1 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-055daa Scaled up replica set payload-processing-e2e-ipp-b-055daa-5f8b69cb54 from 0 to 1 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-055daa-5f8b69cb54 Error creating: pods "payload-processing-e2e-ipp-b-055daa-5f8b69cb54-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-055daa: serviceaccount "payload-processing-e2e-ipp-b-055daa" not found 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-055daa-5f8b69cb54 Created pod: payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc Error creating: pods "payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-055daa: serviceaccount "payload-processing-e2e-ipp-b-055daa" not found 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc Created pod: payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 19m Normal Started pod/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v Container started 19m Normal Started pod/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 Container started 19m Normal Created pod/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 Container created 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 Add eth0 [10.132.0.173/23] from ovn-kubernetes 19m Normal AddedInterface pod/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v Add eth0 [10.133.1.37/23] from ovn-kubernetes 19m Normal Pulled pod/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Created pod/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v Container created 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d Created pod: payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d Error creating: pods "payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cf605a: serviceaccount "payload-processing-e2e-ipp-a-cf605a" not found 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c Error creating: pods "payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cf605a: serviceaccount "payload-processing-e2e-ipp-a-cf605a" not found 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-cf605a Scaled up replica set payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d from 0 to 1 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c Created pod: payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-cf605a Scaled up replica set payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c from 0 to 1 19m Normal Created pod/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 Container created 19m Normal Started pod/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 Container started 19m Normal Started pod/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 Container started 19m Normal Created pod/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 Container created 19m Normal AddedInterface pod/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 Add eth0 [10.132.0.175/23] from ovn-kubernetes 19m Normal Pulled pod/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 Add eth0 [10.132.0.174/23] from ovn-kubernetes 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 19m Normal Killing pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-shared-a-w1-ac7936-openshift-default-66c9fd8c5-g2q6p Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-shared-b-w3-8e2727-openshift-default-9889b484f-jn4jx Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-ac7936-6dbb48c479-zkvg5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5 Created pod: e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-874ab5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 19m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-874ab5-openshift-default Scaled up replica set e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5 from 2 to 3 19m Normal Started pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Container started 19m Normal Created pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Container created 19m Normal Pulled pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Add eth0 [10.132.0.176/23] from ovn-kubernetes 19m Normal Killing pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Stopping container payload-pre-processing 19m Warning Unhealthy pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 19m Warning ProbeError pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-ac7936-58cdbd6fbd-plvrg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Stopping container payload-processing 19m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 2; reason: All metrics below target 19m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-c9ffffc7f from 3 to 2 19m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-c9ffffc7f Deleted pod: maas-default-gateway-openshift-default-c9ffffc7f-fp9dv 19m Normal Killing pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Stopping container istio-proxy 19m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-fp9dv Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-9d3552-656ddb4d4c-pgmld Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-9d3552-56c5d44b9b-5nxcz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-055daa-d488dc8bc-86gn5 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-055daa-5f8b69cb54-5hk6v Stopping container payload-processing 19m Normal Killing pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-jktlh Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-b-w1-9d3552-openshift-default-758cc86dd7-2l5dr Stopping container istio-proxy 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-d9f3ac-b564b45c9-78h5q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w3-d9f3ac-745464ddc6-d5jzn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-cf605a-7b9f9d8b8c-xxdr7 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-cf605a-6dc95c5c5d-6mzq6 Stopping container payload-processing 18m Warning ProbeError pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Readiness probe error: Get "http://10.133.1.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 18m Normal Created pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Container created 18m Normal Started pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Container started 18m Warning Unhealthy pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Readiness probe failed: Get "http://10.133.1.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 18m Normal Pulled pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Pulled pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 18m Normal Killing pod/payload-processing-e2e-worker-w1-e11f03-855bbc8d7b-t87kc Stopping container payload-processing 18m Normal Created pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Container created 18m Normal Started pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Container started 18m Normal Killing pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Stopping container payload-pre-processing 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-055daa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-055daa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning ProbeError pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Startup probe error: Get "http://10.133.1.35:15021/healthz/ready": dial tcp 10.133.1.35:15021: connect: connection refused... 18m Warning Unhealthy pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Startup probe failed: Get "http://10.133.1.35:15021/healthz/ready": dial tcp 10.133.1.35:15021: connect: connection refused 18m Normal Killing pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Stopping container payload-pre-processing 18m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-ph4xb Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w3-d9f3ac-openshift-default-6c75b9cb5f-jj6b2 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning ProbeError pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Startup probe error: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused... 18m Warning Unhealthy pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Startup probe failed: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-055daa Scaled up replica set payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6 from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-055daa Scaled up replica set payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6 Created pod: payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6 Error creating: pods "payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-055daa: serviceaccount "payload-processing-e2e-ipp-b-055daa" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c Created pod: payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c Error creating: pods "payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-055daa: serviceaccount "payload-processing-e2e-ipp-b-055daa" not found 18m Normal Created pod/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j Container created 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv Add eth0 [10.132.0.177/23] from ovn-kubernetes 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv Container created 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv Container started 18m Normal Started pod/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j Container started 18m Normal Pulled pod/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j Add eth0 [10.132.0.178/23] from ovn-kubernetes 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-cf605a Scaled up replica set payload-processing-e2e-ipp-a-cf605a-788659d948 from 0 to 1 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f Created pod: payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-cf605a-788659d948 Error creating: pods "payload-processing-e2e-ipp-a-cf605a-788659d948-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cf605a: serviceaccount "payload-processing-e2e-ipp-a-cf605a" not found 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-cf605a Scaled up replica set payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f from 0 to 1 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f Error creating: pods "payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-cf605a: serviceaccount "payload-processing-e2e-ipp-a-cf605a" not found 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf Add eth0 [10.132.0.179/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf Container started 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-cf605a-788659d948 Created pod: payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 18m Normal AddedInterface pod/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 Add eth0 [10.132.0.180/23] from ovn-kubernetes 18m Normal Pulled pod/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 Container created 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 18m Normal Started pod/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 Container started 18m Normal Created pod/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf Container created 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-4b5459-6bf7866d9f-qd6b5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-4b5459-56c5cd7f4b-rtccx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-cf605a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-cf605a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-e11f03-6c6749f4f5-j7k4t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal ScalingReplicaSet deployment/e2e-ipp-b-3f72d5-openshift-default Scaled up replica set e2e-ipp-b-3f72d5-openshift-default-7df4497f55 from 0 to 1 18m Normal AddedLabel gateway/e2e-ipp-b-3f72d5 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-3f72d5 18m Normal SuccessfulCreate replicaset/e2e-ipp-b-3f72d5-openshift-default-7df4497f55 Created pod: e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g 18m Normal Killing pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-7hhtl Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Stopping container istio-proxy 18m Normal AddedInterface pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Add eth0 [10.132.0.181/23] from ovn-kubernetes 18m Warning Unhealthy pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-055daa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-055daa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 18m Warning ProbeError pod/e2e-shared-a-w6-4b5459-openshift-default-5dfb76cbfc-btjl6 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-3f72d5 Scaled up replica set payload-processing-e2e-ipp-b-3f72d5-6bf8d76564 from 0 to 1 18m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-3f72d5 Scaled up replica set payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9 from 0 to 1 18m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt Add eth0 [10.134.1.1/23] from ovn-kubernetes 18m Normal Started pod/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt Container started 18m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564 Error creating: pods "payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3f72d5: serviceaccount "payload-processing-e2e-ipp-b-3f72d5" not found 18m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9 Error creating: pods "payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3f72d5: serviceaccount "payload-processing-e2e-ipp-b-3f72d5" not found 18m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9 Created pod: payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt 18m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564 Created pod: payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf 18m Normal Created pod/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt Container created 18m Normal AddedInterface pod/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf Add eth0 [10.134.1.2/23] from ovn-kubernetes 18m Normal Pulled pod/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 18m Normal Created pod/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf Container created 18m Normal Started pod/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf Container started 18m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-cf605a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-cf605a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 18m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e9528f-5d6d9745b7-6tr4b Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-a-e9528f-65b447dd7b-q7kv4 Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Stopping container payload-processing 18m Normal Killing pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-r2t9m Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w1-e11f03-openshift-default-84c45ddbd7-mnlr9 Stopping container istio-proxy 17m Warning ProbeError pod/e2e-ipp-a-e9528f-openshift-default-5c7c76bd9c-pwjm6 Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-b-w6-8b0787-openshift-default-66b56f74b5-4vpg9 Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-8b0787-769b97b9fd-l6gx7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-8b0787-7f6dc7b6c5-pbpfh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Stopping container payload-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-055daa-6b6cd4ccf6-dsp8j Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-055daa-6cb9d6b46c-s8mqv Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-ipp-b-3f72d5-6bf8d76564-mm2wf Stopping container payload-processing 17m Normal Killing pod/payload-pre-processing-e2e-ipp-b-3f72d5-669b745cf9-v94mt Stopping container payload-pre-processing 17m Normal Killing pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Stopping container istio-proxy 17m Warning FailedUpdateStatus horizontalpodautoscaler/e2e-ipp-b-055daa-openshift-default horizontalpodautoscalers.autoscaling "e2e-ipp-b-055daa-openshift-default" not found 17m Warning ProbeError pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Readiness probe error: Get "http://10.132.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning ProbeError pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-ipp-b-055daa-openshift-default-68c775fb45-pdrd2 Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning Unhealthy pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Readiness probe failed: Get "http://10.132.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Normal Created pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Container created 17m Normal Pulled pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Container started 17m Warning ProbeError pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Startup probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 17m Warning Unhealthy pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Startup probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-32bad3-d7ffdf995-9qz7m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-worker-w6-32bad3-56784c8f76-7s7pl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Stopping container payload-processing 17m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99 Created pod: payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 17m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99 Error creating: pods "payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3f72d5: serviceaccount "payload-processing-e2e-ipp-b-3f72d5" not found 17m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-3f72d5 Scaled up replica set payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99 from 0 to 1 17m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b Created pod: payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk 17m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-3f72d5 Scaled up replica set payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b from 0 to 1 17m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b Error creating: pods "payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-3f72d5: serviceaccount "payload-processing-e2e-ipp-b-3f72d5" not found 17m Normal Created pod/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 Container created 17m Normal AddedInterface pod/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk Add eth0 [10.132.0.184/23] from ovn-kubernetes 17m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 Add eth0 [10.132.0.183/23] from ovn-kubernetes 17m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Started pod/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 Container started 17m Normal Started pod/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk Container started 17m Normal Created pod/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk Container created 17m Normal Pulled pod/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 17m Normal Killing pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Stopping container istio-proxy 17m Normal Killing pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Stopping container istio-proxy 17m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-3f72d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-3f72d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 17m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-b4c9f Stopping container istio-proxy 17m Warning ProbeError pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Readiness probe error: HTTP probe failed with statuscode: 503... 17m Normal Killing pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-krvxh Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Readiness probe error: Get "http://10.133.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/e2e-shared-a-w5-6b5f19-openshift-default-85665454c-q4x8d Readiness probe failed: Get "http://10.133.0.147:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-6b5f19-cbc474456-lgj2d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-worker-w6-32bad3-openshift-default-6cc8655c5-2dvtp Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-6b5f19-6ddd55fb5f-djmj9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Stopping container payload-pre-processing 16m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff Deleted pod: e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 16m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-ae1dd9-openshift-default New size: 1; reason: All metrics below target 16m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-ae1dd9-openshift-default Scaled down replica set e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff from 2 to 1 16m Normal Killing pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-9qth4 Stopping container istio-proxy 16m Normal Killing pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Stopping container istio-proxy 16m Warning Unhealthy pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Readiness probe failed: HTTP probe failed with statuscode: 503 16m Warning ProbeError pod/e2e-shared-b-w5-ae1dd9-openshift-default-668c46fcff-fvxzb Readiness probe error: HTTP probe failed with statuscode: 503... 16m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-ae1dd9-55bb58fb95-c82n9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-ae1dd9-5b9d84b87b-24ftm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Normal Killing pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Stopping container payload-pre-processing 16m Normal Killing pod/payload-processing-e2e-ipp-a-cf605a-788659d948-6j8m8 Stopping container payload-processing 16m Normal Killing pod/payload-pre-processing-e2e-ipp-a-cf605a-95c4f479f-hmpdf Stopping container payload-pre-processing 16m Warning Unhealthy pod/payload-processing-e2e-worker-w5-f7d1a1-7dcbd5df7b-clmq9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 16m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-f7d1a1-55d6db6f9f-9dd5z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 15m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-wfmmn Stopping container istio-proxy 15m Normal Killing pod/e2e-worker-w5-f7d1a1-openshift-default-7fcf76dfc9-rbgnd Stopping container istio-proxy 15m Normal Killing pod/e2e-ipp-a-cf605a-openshift-default-79f69d44f-xggxg Stopping container istio-proxy 15m Normal Killing pod/payload-processing-e2e-ipp-b-3f72d5-5c784c8d8b-v7svk Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-ipp-b-3f72d5-fc95f6f99-hn565 Stopping container payload-pre-processing 15m Normal Killing pod/e2e-ipp-b-3f72d5-openshift-default-7df4497f55-d5r7g Stopping container istio-proxy 15m Normal Killing pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Stopping container payload-processing 15m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Stopping container payload-pre-processing 14m Normal Killing pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-dvw42 Stopping container istio-proxy 14m Warning Unhealthy pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-a-w0-d823d1-openshift-default-566cfb7479-tdn2l Readiness probe error: HTTP probe failed with statuscode: 503... 14m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-d823d1-5c9656fc48-qj5cf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-d823d1-57f4d97b55-bwhz9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Normal Killing pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Stopping container payload-processing 14m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Stopping container payload-pre-processing 14m Normal Killing pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-jxjdb Stopping container istio-proxy 14m Normal Killing pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-kh9rf Stopping container istio-proxy 14m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 14m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 14m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-mhnwk 14m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-mhnwk Stopping container discovery 14m Warning Unhealthy pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Readiness probe failed: HTTP probe failed with statuscode: 503 14m Warning ProbeError pod/e2e-shared-b-w0-874ab5-openshift-default-554c6b8cc5-hl6pt Readiness probe error: HTTP probe failed with statuscode: 503... 14m Normal Killing pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Stopping container payload-pre-processing 14m Normal Killing pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Stopping container payload-processing 14m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-874ab5-fbc9f8659-mxzrn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-874ab5-79d6bdcf7b-q4857 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-aeff1d-5f7d4bc7d7-qs76z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 14m Warning Unhealthy pod/payload-processing-e2e-worker-w0-aeff1d-77d496d48d-khlt7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 13m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-fkm5w Stopping container istio-proxy 13m Normal Killing pod/e2e-worker-w0-aeff1d-openshift-default-5bcf8b7867-9kz2r Stopping container istio-proxy 13m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Container started 13m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Container created 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-c22dw 13m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Container started 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Add eth0 [10.132.0.188/23] from ovn-kubernetes 13m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Container created 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Add eth0 [10.134.1.22/23] from ovn-kubernetes 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 2 to 4 13m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Startup probe error: Get "http://10.134.1.22:15021/healthz/ready": dial tcp 10.134.1.22:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Startup probe failed: Get "http://10.134.1.22:15021/healthz/ready": dial tcp 10.134.1.22:15021: connect: connection refused 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Startup probe failed: Get "http://10.134.1.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Startup probe error: Get "http://10.134.1.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Container created 13m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 4 to 7 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-x5p4f 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Add eth0 [10.133.1.47/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-qv642 13m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Container started 13m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Container created 13m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Container created 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Add eth0 [10.134.1.23/23] from ovn-kubernetes 13m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 13m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 13m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Add eth0 [10.132.0.189/23] from ovn-kubernetes 13m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Container started 13m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 13m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Container started 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Startup probe error: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Startup probe error: Get "http://10.134.1.23:15021/healthz/ready": dial tcp 10.134.1.23:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-x5p4f Startup probe failed: Get "http://10.134.1.23:15021/healthz/ready": dial tcp 10.134.1.23:15021: connect: connection refused 13m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 13m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-qv642 Startup probe failed: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused 12m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-rl5mn 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-8bc4f 12m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-c9ffffc7f Created pod: maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 12m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-c9ffffc7f from 7 to 10 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Add eth0 [10.132.0.190/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Container created 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Add eth0 [10.133.1.50/23] from ovn-kubernetes 12m Normal AddedInterface pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Add eth0 [10.134.1.24/23] from ovn-kubernetes 12m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Pulled pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 12m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Container started 12m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Container created 12m Normal Created pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Container created 12m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Container started 12m Normal Started pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Container started 12m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Startup probe error: Get "http://10.132.0.190:15021/healthz/ready": dial tcp 10.132.0.190:15021: connect: connection refused... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Startup probe error: Get "http://10.134.1.24:15021/healthz/ready": dial tcp 10.134.1.24:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Startup probe failed: Get "http://10.134.1.24:15021/healthz/ready": dial tcp 10.134.1.24:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Startup probe error: Get "http://10.133.1.50:15021/healthz/ready": dial tcp 10.133.1.50:15021: connect: connection refused... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Startup probe failed: Get "http://10.132.0.190:15021/healthz/ready": dial tcp 10.132.0.190:15021: connect: connection refused 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Startup probe failed: Get "http://10.133.1.50:15021/healthz/ready": dial tcp 10.133.1.50:15021: connect: connection refused 12m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Startup probe error: Get "http://10.133.1.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Startup probe failed: Get "http://10.134.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Startup probe failed: Get "http://10.132.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Startup probe error: Get "http://10.132.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Startup probe error: Get "http://10.134.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-rl5mn Startup probe failed: Get "http://10.133.1.50:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 9m42s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Readiness probe failed: Get "http://10.133.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 9m42s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-xmv89 Readiness probe error: Get "http://10.133.0.157:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 9m2s Normal AddedLabel gateway/e2e-worker-single-main-ed4e18 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-ed4e18 9m2s Normal ScalingReplicaSet deployment/e2e-worker-single-main-ed4e18-openshift-default Scaled up replica set e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f from 0 to 1 9m1s Normal AddedInterface pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Add eth0 [10.133.1.61/23] from ovn-kubernetes 9m1s Normal SuccessfulCreate replicaset/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f Created pod: e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l 8m56s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-ed4e18-6585ddfbcf Error creating: pods "payload-pre-processing-e2e-worker-single-main-ed4e18-6585ddfbcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ed4e18: serviceaccount "payload-processing-e2e-worker-single-main-ed4e18" not found 8m56s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6b Created pod: payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 8m56s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-ed4e18-6585ddfbcf Created pod: payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs 8m56s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-ed4e18 Scaled up replica set payload-pre-processing-e2e-worker-single-main-ed4e18-6585ddfbcf from 0 to 1 8m56s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-ed4e18 Scaled up replica set payload-processing-e2e-worker-single-main-ed4e18-78d5856c6b from 0 to 1 8m56s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6b Error creating: pods "payload-processing-e2e-worker-single-main-ed4e18-78d5856c6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ed4e18: serviceaccount "payload-processing-e2e-worker-single-main-ed4e18" not found 8m55s Normal Started pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Container started 8m55s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Add eth0 [10.133.1.62/23] from ovn-kubernetes 8m55s Normal Pulled pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m55s Normal Created pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Container created 8m55s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m55s Normal Created pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Container created 8m55s Normal Started pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Container started 8m55s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Add eth0 [10.132.0.192/23] from ovn-kubernetes 8m44s Warning Unhealthy pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Readiness probe failed: Get "http://10.133.1.61:15021/healthz/ready": read tcp 10.133.0.2:43490->10.133.1.61:15021: read: connection reset by peer 8m44s Warning ProbeError pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Readiness probe error: Get "http://10.133.1.61:15021/healthz/ready": read tcp 10.133.0.2:43490->10.133.1.61:15021: read: connection reset by peer... 8m43s Normal Pulled pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 8m43s Normal Started pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Container started 8m43s Normal Created pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Container created 8m38s Warning Unhealthy pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Startup probe failed: Get "http://10.133.1.61:15021/healthz/ready": dial tcp 10.133.1.61:15021: connect: connection refused 8m38s Warning ProbeError pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Startup probe error: Get "http://10.133.1.61:15021/healthz/ready": dial tcp 10.133.1.61:15021: connect: connection refused... 8m1s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-ed4e18-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 8m1s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-ed4e18-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m55s Normal Killing pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Stopping container payload-processing 7m55s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Stopping container payload-pre-processing 7m46s Normal Killing pod/e2e-worker-single-main-ed4e18-openshift-default-6d88cfdc4f2vl4l Stopping container istio-proxy 7m34s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-ed4e18-6585dqjwjs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 7m33s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-ed4e18-78d5856c6xkw96 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m31s Normal AddedLabel gateway/e2e-worker-single-main-70abcb Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-70abcb 5m31s Normal SuccessfulCreate replicaset/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49 Created pod: e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 5m31s Normal AddedInterface pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Add eth0 [10.133.1.69/23] from ovn-kubernetes 5m31s Normal ScalingReplicaSet deployment/e2e-worker-single-main-70abcb-openshift-default Scaled up replica set e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49 from 0 to 1 5m26s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-70abcb Scaled up replica set payload-processing-e2e-worker-single-main-70abcb-64bd795f74 from 0 to 1 5m26s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-70abcb Scaled up replica set payload-pre-processing-e2e-worker-single-main-70abcb-649f95f45f from 0 to 1 5m26s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-70abcb-649f95f45f Created pod: payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz 5m26s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-70abcb-649f95f45f Error creating: pods "payload-pre-processing-e2e-worker-single-main-70abcb-649f95f45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-70abcb: serviceaccount "payload-processing-e2e-worker-single-main-70abcb" not found 5m26s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-70abcb-64bd795f74 Error creating: pods "payload-processing-e2e-worker-single-main-70abcb-64bd795f74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-70abcb: serviceaccount "payload-processing-e2e-worker-single-main-70abcb" not found 5m26s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-70abcb-64bd795f74 Created pod: payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c 5m25s Normal Created pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Container created 5m25s Normal Started pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Container started 5m25s Normal Started pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Container started 5m25s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m25s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Add eth0 [10.132.0.195/23] from ovn-kubernetes 5m25s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Add eth0 [10.133.1.70/23] from ovn-kubernetes 5m25s Normal Pulled pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m25s Normal Created pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Container created 5m13s Warning ProbeError pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Readiness probe error: Get "http://10.133.1.69:15021/healthz/ready": read tcp 10.133.0.2:43068->10.133.1.69:15021: read: connection reset by peer... 5m13s Warning Unhealthy pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Readiness probe failed: Get "http://10.133.1.69:15021/healthz/ready": read tcp 10.133.0.2:43068->10.133.1.69:15021: read: connection reset by peer 5m12s Normal Created pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Container created 5m12s Normal Pulled pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m12s Normal Started pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Container started 5m7s Warning Unhealthy pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Startup probe failed: Get "http://10.133.1.69:15021/healthz/ready": dial tcp 10.133.1.69:15021: connect: connection refused 5m7s Warning ProbeError pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Startup probe error: Get "http://10.133.1.69:15021/healthz/ready": dial tcp 10.133.1.69:15021: connect: connection refused... 4m31s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-70abcb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m31s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-70abcb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m29s Normal Killing pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Stopping container payload-processing 4m29s Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-worker-single-main-70abcb Failed to update endpoint openshift-ingress/payload-processing-e2e-worker-single-main-70abcb: Operation cannot be fulfilled on endpoints "payload-processing-e2e-worker-single-main-70abcb": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-worker-single-main-70abcb, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 3f6653f6-5da7-4652-9e7f-f1690fdfde02, UID in object meta: 4m29s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Stopping container payload-pre-processing 4m21s Normal Killing pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Stopping container istio-proxy 4m20s Warning Unhealthy pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Readiness probe failed: HTTP probe failed with statuscode: 503 4m20s Warning ProbeError pod/e2e-worker-single-main-70abcb-openshift-default-5d58c5bc49qtt74 Readiness probe error: HTTP probe failed with statuscode: 503... 4m13s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-70abcb-649f95ghgz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 4m13s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-70abcb-64bd795f7xs97c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 3m38s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Readiness probe failed: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m38s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-pnfx8 Readiness probe error: Get "http://10.132.0.189:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m21s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Readiness probe failed: Get "http://10.134.1.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m21s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-c22dw Readiness probe error: Get "http://10.134.1.22:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m21s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Readiness probe failed: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m21s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Readiness probe error: Get "http://10.134.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m21s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-4nnf7 Readiness probe error: Get "http://10.132.0.188:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m21s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-8bc4f Readiness probe failed: Get "http://10.134.1.24:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 3m21s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Readiness probe error: Get "http://10.132.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 3m21s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-cj4p7 Readiness probe failed: Get "http://10.132.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m29s Warning Unhealthy pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m29s Warning ProbeError pod/maas-default-gateway-openshift-default-c9ffffc7f-54ldl Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...