=== events in openshift-ingress (2026-11-01T16:54:00Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q to ip-10-0-128-238.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 to ip-10-0-143-101.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Successfully assigned openshift-ingress/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc to ip-10-0-128-238.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np to ip-10-0-128-238.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt to ip-10-0-143-101.us-east-2.compute.internal 38m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 to ip-10-0-143-101.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Successfully assigned openshift-ingress/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 to ip-10-0-128-238.us-east-2.compute.internal 7m59s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k to ip-10-0-141-145.us-east-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Successfully assigned openshift-ingress/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 to ip-10-0-143-101.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf to ip-10-0-128-238.us-east-2.compute.internal 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm to ip-10-0-141-145.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq to ip-10-0-143-101.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx to ip-10-0-128-238.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Successfully assigned openshift-ingress/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk to ip-10-0-128-238.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Successfully assigned openshift-ingress/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q to ip-10-0-143-101.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq Successfully assigned openshift-ingress/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc to ip-10-0-128-238.us-east-2.compute.internal 8m3s Normal Scheduled pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Successfully assigned openshift-ingress/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 to ip-10-0-128-238.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl to ip-10-0-128-238.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Successfully assigned openshift-ingress/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv to ip-10-0-143-101.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 to ip-10-0-141-145.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm Successfully assigned openshift-ingress/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq to ip-10-0-141-145.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Successfully assigned openshift-ingress/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 to ip-10-0-143-101.us-east-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq to ip-10-0-128-238.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Successfully assigned openshift-ingress/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz Successfully assigned openshift-ingress/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r to ip-10-0-143-101.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p to ip-10-0-141-145.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Successfully assigned openshift-ingress/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl to ip-10-0-128-238.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt to ip-10-0-143-101.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z to ip-10-0-141-145.us-east-2.compute.internal 5m14s Normal Scheduled pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Successfully assigned openshift-ingress/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c to ip-10-0-128-238.us-east-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p to ip-10-0-128-238.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Successfully assigned openshift-ingress/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b to ip-10-0-128-238.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn to ip-10-0-143-101.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Successfully assigned openshift-ingress/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Successfully assigned openshift-ingress/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc to ip-10-0-128-238.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz Successfully assigned openshift-ingress/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz to ip-10-0-141-145.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd to ip-10-0-128-238.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Successfully assigned openshift-ingress/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv to ip-10-0-143-101.us-east-2.compute.internal 31m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 to ip-10-0-141-145.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt to ip-10-0-141-145.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Successfully assigned openshift-ingress/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p to ip-10-0-128-238.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 to ip-10-0-143-101.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Successfully assigned openshift-ingress/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg Successfully assigned openshift-ingress/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg to ip-10-0-128-238.us-east-2.compute.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck to ip-10-0-128-238.us-east-2.compute.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 to ip-10-0-128-238.us-east-2.compute.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq to ip-10-0-128-238.us-east-2.compute.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 to ip-10-0-141-145.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 to ip-10-0-128-238.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Successfully assigned openshift-ingress/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r to ip-10-0-141-145.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc to ip-10-0-128-238.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x to ip-10-0-141-145.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv to ip-10-0-141-145.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Successfully assigned openshift-ingress/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w to ip-10-0-128-238.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l to ip-10-0-141-145.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps to ip-10-0-141-145.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Successfully assigned openshift-ingress/e2e-api-a-166778-openshift-default-869f777c67-4ddgh to ip-10-0-141-145.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj to ip-10-0-141-145.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm to ip-10-0-141-145.us-east-2.compute.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j to ip-10-0-141-145.us-east-2.compute.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 to ip-10-0-128-238.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 to ip-10-0-128-238.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 to ip-10-0-141-145.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz to ip-10-0-141-145.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c to ip-10-0-141-145.us-east-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-whjkj to ip-10-0-143-101.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr to ip-10-0-128-238.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Successfully assigned openshift-ingress/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl to ip-10-0-128-238.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Successfully assigned openshift-ingress/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p to ip-10-0-141-145.us-east-2.compute.internal 61m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll to ip-10-0-143-101.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 53m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq to ip-10-0-143-101.us-east-2.compute.internal 45m Normal Preempted pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Preempted by pod 3d6c2dce-ecb9-4ca8-96a4-c2e0a80d6ca2 on node ip-10-0-141-145.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-vhjzm to ip-10-0-143-101.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c to ip-10-0-141-145.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs to ip-10-0-128-238.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Successfully assigned openshift-ingress/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b to ip-10-0-143-101.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf to ip-10-0-143-101.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Successfully assigned openshift-ingress/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b to ip-10-0-143-101.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 to ip-10-0-141-145.us-east-2.compute.internal 37m Warning FailedScheduling pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 53m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt to ip-10-0-128-238.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Successfully assigned openshift-ingress/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq to ip-10-0-128-238.us-east-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-v67pm to ip-10-0-141-145.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 to ip-10-0-141-145.us-east-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d to ip-10-0-128-238.us-east-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg to ip-10-0-128-238.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Successfully assigned openshift-ingress/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 to ip-10-0-141-145.us-east-2.compute.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-s62zq to ip-10-0-143-101.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-7b98bb678c-44k4f Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-44k4f to ip-10-0-141-145.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Successfully assigned openshift-ingress/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt to ip-10-0-141-145.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s to ip-10-0-141-145.us-east-2.compute.internal 37m Warning FailedScheduling pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 54m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd to ip-10-0-141-145.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Successfully assigned openshift-ingress/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h to ip-10-0-128-238.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn to ip-10-0-143-101.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr to ip-10-0-128-238.us-east-2.compute.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-r9wbx to ip-10-0-141-145.us-east-2.compute.internal 20m Normal Scheduled pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Successfully assigned openshift-ingress/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d to ip-10-0-128-238.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Successfully assigned openshift-ingress/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m to ip-10-0-141-145.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Successfully assigned openshift-ingress/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm to ip-10-0-128-238.us-east-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Successfully assigned openshift-ingress/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp Successfully assigned openshift-ingress/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp to ip-10-0-143-101.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db to ip-10-0-141-145.us-east-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-r5m84 to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Successfully assigned openshift-ingress/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 to ip-10-0-143-101.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 to ip-10-0-141-145.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr to ip-10-0-141-145.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m to ip-10-0-128-238.us-east-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq to ip-10-0-141-145.us-east-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-q757x to ip-10-0-128-238.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j to ip-10-0-128-238.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Successfully assigned openshift-ingress/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 to ip-10-0-141-145.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s to ip-10-0-141-145.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Successfully assigned openshift-ingress/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 to ip-10-0-143-101.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Successfully assigned openshift-ingress/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n to ip-10-0-128-238.us-east-2.compute.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-lfl9n to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd to ip-10-0-143-101.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Successfully assigned openshift-ingress/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv to ip-10-0-128-238.us-east-2.compute.internal 39m Warning FailedScheduling pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-cvl2d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 47m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m to ip-10-0-141-145.us-east-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq to ip-10-0-141-145.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws to ip-10-0-141-145.us-east-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-klsmb to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx to ip-10-0-141-145.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 to ip-10-0-143-101.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Successfully assigned openshift-ingress/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f to ip-10-0-141-145.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Successfully assigned openshift-ingress/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 to ip-10-0-128-238.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 to ip-10-0-141-145.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 to ip-10-0-141-145.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Successfully assigned openshift-ingress/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg to ip-10-0-128-238.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 to ip-10-0-141-145.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Successfully assigned openshift-ingress/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws to ip-10-0-128-238.us-east-2.compute.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-hmxgm to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj to ip-10-0-143-101.us-east-2.compute.internal 33m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb to ip-10-0-141-145.us-east-2.compute.internal 44m Warning FailedScheduling pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Warning FailedScheduling pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 44m Normal Scheduled pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Successfully assigned openshift-ingress/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw to ip-10-0-128-238.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq to ip-10-0-141-145.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Successfully assigned openshift-ingress/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 to ip-10-0-143-101.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Successfully assigned openshift-ingress/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg to ip-10-0-143-101.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 54m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp to ip-10-0-143-101.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-hfzvl to ip-10-0-143-101.us-east-2.compute.internal 31m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 to ip-10-0-141-145.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 to ip-10-0-141-145.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 to ip-10-0-128-238.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc to ip-10-0-128-238.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx to ip-10-0-128-238.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr to ip-10-0-143-101.us-east-2.compute.internal 15m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-gnmrk to ip-10-0-141-145.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Successfully assigned openshift-ingress/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 to ip-10-0-141-145.us-east-2.compute.internal 4m42s Normal Scheduled pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Successfully assigned openshift-ingress/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx to ip-10-0-143-101.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b to ip-10-0-141-145.us-east-2.compute.internal 66m Normal Scheduled pod/router-default-69f6694997-vvcm2 Successfully assigned openshift-ingress/router-default-69f6694997-vvcm2 to ip-10-0-143-101.us-east-2.compute.internal 66m Warning FailedScheduling pod/router-default-69f6694997-vvcm2 no nodes available to schedule pods 73m Warning FailedScheduling pod/router-default-69f6694997-vvcm2 no nodes available to schedule pods 53m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv to ip-10-0-128-238.us-east-2.compute.internal 4m38s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk to ip-10-0-128-238.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Successfully assigned openshift-ingress/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl to ip-10-0-128-238.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc to ip-10-0-143-101.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Successfully assigned openshift-ingress/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 to ip-10-0-141-145.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 55m Normal Scheduled pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs to ip-10-0-141-145.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Successfully assigned openshift-ingress/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz to ip-10-0-143-101.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg to ip-10-0-143-101.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Successfully assigned openshift-ingress/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 to ip-10-0-128-238.us-east-2.compute.internal 38m Warning FailedScheduling pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 55m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k to ip-10-0-141-145.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Successfully assigned openshift-ingress/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv to ip-10-0-128-238.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s to ip-10-0-143-101.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn to ip-10-0-141-145.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Successfully assigned openshift-ingress/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 to ip-10-0-128-238.us-east-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc to ip-10-0-143-101.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 to ip-10-0-128-238.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-55fbbbf9dc-l5qnm Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-l5qnm to ip-10-0-128-238.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Successfully assigned openshift-ingress/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 to ip-10-0-143-101.us-east-2.compute.internal 5m10s Normal Scheduled pod/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k to ip-10-0-141-145.us-east-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-a4240a-745976f465-st69v to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Successfully assigned openshift-ingress/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s to ip-10-0-143-101.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl to ip-10-0-143-101.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Successfully assigned openshift-ingress/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk to ip-10-0-141-145.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Successfully assigned openshift-ingress/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn to ip-10-0-128-238.us-east-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw to ip-10-0-141-145.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Successfully assigned openshift-ingress/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch to ip-10-0-143-101.us-east-2.compute.internal 56m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-784444-55c46475-d78lt to ip-10-0-128-238.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Successfully assigned openshift-ingress/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg to ip-10-0-128-238.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Successfully assigned openshift-ingress/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Successfully assigned openshift-ingress/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Successfully assigned openshift-ingress/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g to ip-10-0-141-145.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr to ip-10-0-128-238.us-east-2.compute.internal 61m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-9rb5l to ip-10-0-143-101.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Successfully assigned openshift-ingress/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c to ip-10-0-141-145.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc to ip-10-0-128-238.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Successfully assigned openshift-ingress/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l to ip-10-0-143-101.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Successfully assigned openshift-ingress/e2e-inf-b-375350-openshift-default-5c64667db7-664vq to ip-10-0-143-101.us-east-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt to ip-10-0-128-238.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Successfully assigned openshift-ingress/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 to ip-10-0-128-238.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs to ip-10-0-128-238.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv to ip-10-0-141-145.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-8s26k to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt to ip-10-0-128-238.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Warning FailedScheduling pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 37m Normal Scheduled pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Successfully assigned openshift-ingress/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 to ip-10-0-128-238.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-7f5667f86c-65xj6 to ip-10-0-128-238.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq to ip-10-0-141-145.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 to ip-10-0-128-238.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg to ip-10-0-141-145.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw to ip-10-0-141-145.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl to ip-10-0-141-145.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 Successfully assigned openshift-ingress/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 to ip-10-0-128-238.us-east-2.compute.internal 4m38s Normal Scheduled pod/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Successfully assigned openshift-ingress/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh Successfully assigned openshift-ingress/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh to ip-10-0-128-238.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 to ip-10-0-143-101.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Successfully assigned openshift-ingress/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 to ip-10-0-141-145.us-east-2.compute.internal 60m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k to ip-10-0-143-101.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq to ip-10-0-141-145.us-east-2.compute.internal 7m59s Normal Scheduled pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp to ip-10-0-128-238.us-east-2.compute.internal 60m Normal Scheduled pod/kube-auth-proxy-6759f4f87c-t6l44 Successfully assigned openshift-ingress/kube-auth-proxy-6759f4f87c-t6l44 to ip-10-0-141-145.us-east-2.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn to ip-10-0-128-238.us-east-2.compute.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s to ip-10-0-128-238.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm to ip-10-0-141-145.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f to ip-10-0-141-145.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps to ip-10-0-143-101.us-east-2.compute.internal 60m Normal Scheduled pod/kube-auth-proxy-6759f4f87c-mhx92 Successfully assigned openshift-ingress/kube-auth-proxy-6759f4f87c-mhx92 to ip-10-0-128-238.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm to ip-10-0-128-238.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Successfully assigned openshift-ingress/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp to ip-10-0-143-101.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr to ip-10-0-128-238.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr to ip-10-0-141-145.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft Successfully assigned openshift-ingress/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Successfully assigned openshift-ingress/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w to ip-10-0-141-145.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b to ip-10-0-128-238.us-east-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs to ip-10-0-128-238.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 to ip-10-0-143-101.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj to ip-10-0-128-238.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Successfully assigned openshift-ingress/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z to ip-10-0-141-145.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r to ip-10-0-128-238.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b Successfully assigned openshift-ingress/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b to ip-10-0-128-238.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Successfully assigned openshift-ingress/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr to ip-10-0-128-238.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h to ip-10-0-143-101.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn to ip-10-0-128-238.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j to ip-10-0-143-101.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h to ip-10-0-128-238.us-east-2.compute.internal 49m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-rtv2k to ip-10-0-141-145.us-east-2.compute.internal 45m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 to ip-10-0-141-145.us-east-2.compute.internal 45m Warning FailedScheduling pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: not eligible due to a terminating pod on the nominated node. 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs to ip-10-0-141-145.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Successfully assigned openshift-ingress/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv to ip-10-0-143-101.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj to ip-10-0-128-238.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Successfully assigned openshift-ingress/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x to ip-10-0-141-145.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph to ip-10-0-128-238.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s Successfully assigned openshift-ingress/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s to ip-10-0-141-145.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx to ip-10-0-143-101.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Successfully assigned openshift-ingress/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc to ip-10-0-143-101.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k to ip-10-0-141-145.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Successfully assigned openshift-ingress/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk to ip-10-0-141-145.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n to ip-10-0-141-145.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm to ip-10-0-141-145.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 to ip-10-0-141-145.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs Successfully assigned openshift-ingress/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs to ip-10-0-128-238.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq to ip-10-0-128-238.us-east-2.compute.internal 45m Warning FailedScheduling pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: not eligible due to a terminating pod on the nominated node. 44m Normal Scheduled pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt to ip-10-0-128-238.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Successfully assigned openshift-ingress/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv to ip-10-0-128-238.us-east-2.compute.internal 45m Warning FailedScheduling pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 0/3 nodes are available: 3 Insufficient cpu. 52m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz to ip-10-0-128-238.us-east-2.compute.internal 45m Warning FailedScheduling pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Warning FailedScheduling pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 45m Normal Scheduled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Successfully assigned openshift-ingress/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc to ip-10-0-141-145.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf to ip-10-0-141-145.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Successfully assigned openshift-ingress/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m to ip-10-0-143-101.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf to ip-10-0-128-238.us-east-2.compute.internal 55m Normal Scheduled pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh to ip-10-0-141-145.us-east-2.compute.internal 59m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-b9q8c Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-b9q8c to ip-10-0-128-238.us-east-2.compute.internal 37m Warning FailedScheduling pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 36m Normal Scheduled pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Successfully assigned openshift-ingress/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 to ip-10-0-141-145.us-east-2.compute.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d to ip-10-0-128-238.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f to ip-10-0-141-145.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg to ip-10-0-128-238.us-east-2.compute.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc to ip-10-0-141-145.us-east-2.compute.internal 5m10s Normal Scheduled pod/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f Successfully assigned openshift-ingress/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f to ip-10-0-128-238.us-east-2.compute.internal 49m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-2qs8x to ip-10-0-143-101.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 to ip-10-0-128-238.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Successfully assigned openshift-ingress/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd to ip-10-0-128-238.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj Successfully assigned openshift-ingress/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj to ip-10-0-128-238.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 to ip-10-0-141-145.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h to ip-10-0-128-238.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Successfully assigned openshift-ingress/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm to ip-10-0-141-145.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Successfully assigned openshift-ingress/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr to ip-10-0-128-238.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 55m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh to ip-10-0-143-101.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 to ip-10-0-141-145.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 to ip-10-0-143-101.us-east-2.compute.internal 52m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-27n7m to ip-10-0-141-145.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg to ip-10-0-141-145.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv Successfully assigned openshift-ingress/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv to ip-10-0-128-238.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh to ip-10-0-128-238.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp to ip-10-0-143-101.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Successfully assigned openshift-ingress/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf to ip-10-0-128-238.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx to ip-10-0-128-238.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Successfully assigned openshift-ingress/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s to ip-10-0-141-145.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 to ip-10-0-128-238.us-east-2.compute.internal 61m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-vl7qf Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-vl7qf to ip-10-0-128-238.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Successfully assigned openshift-ingress/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs to ip-10-0-141-145.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv to ip-10-0-143-101.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 to ip-10-0-128-238.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 to ip-10-0-128-238.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv Successfully assigned openshift-ingress/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv to ip-10-0-141-145.us-east-2.compute.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw to ip-10-0-128-238.us-east-2.compute.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz to ip-10-0-141-145.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Successfully assigned openshift-ingress/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 to ip-10-0-128-238.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg to ip-10-0-128-238.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Successfully assigned openshift-ingress/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df to ip-10-0-141-145.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Successfully assigned openshift-ingress/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z to ip-10-0-143-101.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz to ip-10-0-143-101.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw Successfully assigned openshift-ingress/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw to ip-10-0-128-238.us-east-2.compute.internal 30m Normal Scheduled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Successfully assigned openshift-ingress/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt to ip-10-0-141-145.us-east-2.compute.internal 73m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-69f6694997 from 0 to 1 73m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 73m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 73m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 73m Normal SuccessfulCreate replicaset/router-default-69f6694997 Created pod: router-default-69f6694997-vvcm2 66m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 66m Warning FailedMount pod/router-default-69f6694997-vvcm2 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-69f6694997-vvcm2 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Warning FailedMount pod/router-default-69f6694997-vvcm2 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-69f6694997-vvcm2 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Normal Pulled pod/router-default-69f6694997-vvcm2 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 66m Normal Started pod/router-default-69f6694997-vvcm2 Container started 66m Normal AddedInterface pod/router-default-69f6694997-vvcm2 Add eth0 [10.132.0.17/23] from ovn-kubernetes 66m Normal Created pod/router-default-69f6694997-vvcm2 Container created 61m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 61m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-vl7qf 61m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 61m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-vl7qf Add eth0 [10.134.0.20/23] from ovn-kubernetes 61m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-vl7qf Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 61m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-9rb5l 61m Normal Created pod/istiod-openshift-gateway-7876bfcb74-vl7qf Container created 61m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 61m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-65dccbd94c Created pod: openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll 61m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-65dccbd94c from 0 to 1 61m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 0 to 1 61m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-vl7qf Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 2.958s (2.958s including waiting). Image size: 185476477 bytes. 61m Normal Started pod/istiod-openshift-gateway-7876bfcb74-vl7qf Container started 61m Normal Pulling pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Add eth0 [10.132.0.33/23] from ovn-kubernetes 61m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 61m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll Add eth0 [10.132.0.34/23] from ovn-kubernetes 61m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 61m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.347s (8.347s including waiting). Image size: 183555333 bytes. 61m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.708s (8.708s including waiting). Image size: 183555333 bytes. 61m Normal Created pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll Container created 61m Normal Started pod/openshift-ai-inference-openshift-ai-inference-65dccbd94c-shrll Container started 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-6759f4f87c from 0 to 2 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-6759f4f87c Created pod: kube-auth-proxy-6759f4f87c-mhx92 60m Normal SuccessfulCreate replicaset/kube-auth-proxy-6759f4f87c Created pod: kube-auth-proxy-6759f4f87c-t6l44 60m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-7cfc97f8f7 from 0 to 1 60m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-7cfc97f8f7 Created pod: data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k 60m Normal AddedInterface pod/kube-auth-proxy-6759f4f87c-t6l44 Add eth0 [10.133.0.26/23] from ovn-kubernetes 60m Normal AddedInterface pod/kube-auth-proxy-6759f4f87c-mhx92 Add eth0 [10.134.0.22/23] from ovn-kubernetes 60m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k Add eth0 [10.132.0.40/23] from ovn-kubernetes 60m Normal Started pod/data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k Container started 60m Normal Pulling pod/kube-auth-proxy-6759f4f87c-t6l44 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal Pulling pod/kube-auth-proxy-6759f4f87c-mhx92 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 60m Normal Created pod/data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k Container created 60m Normal Pulled pod/data-science-gateway-data-science-gateway-class-7cfc97f8f76mh9k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/kube-auth-proxy-6759f4f87c-mhx92 Container created 60m Normal Started pod/kube-auth-proxy-6759f4f87c-mhx92 Container started 60m Normal Pulled pod/kube-auth-proxy-6759f4f87c-mhx92 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.564s (2.565s including waiting). Image size: 188904677 bytes. 60m Normal Pulled pod/kube-auth-proxy-6759f4f87c-t6l44 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 4.519s (4.519s including waiting). Image size: 188904677 bytes. 60m Normal Created pod/kube-auth-proxy-6759f4f87c-t6l44 Container created 60m Normal Started pod/kube-auth-proxy-6759f4f87c-t6l44 Container started 60m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-b9q8c 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 59m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-b9q8c Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-b9q8c Add eth0 [10.134.0.25/23] from ovn-kubernetes 59m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-b9q8c Container started 59m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-b9q8c Container created 59m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 59m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-vl7qf Stopping container discovery 59m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-vl7qf 58m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Container created 58m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Container started 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Startup probe failed: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Startup probe error: Get "http://10.132.0.33:15021/healthz/ready": dial tcp 10.132.0.33:15021: connect: connection refused... 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 58m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 58m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-8s26k 57m Normal Pulling pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Add eth0 [10.132.0.55/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-kkcz5 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Add eth0 [10.134.0.32/23] from ovn-kubernetes 57m Normal Pulling pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 57m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Container created 57m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Container started 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-hfzvl 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Add eth0 [10.133.0.36/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 1 to 4 57m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.246s (3.246s including waiting). Image size: 183555333 bytes. 57m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Container created 57m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Container started 57m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Container created 57m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.223s (3.223s including waiting). Image size: 183555333 bytes. 57m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Container started 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Startup probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Startup probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 57m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Startup probe error: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Startup probe failed: Get "http://10.133.0.36:15021/healthz/ready": dial tcp 10.133.0.36:15021: connect: connection refused 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Startup probe failed: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Startup probe error: Get "http://10.134.0.32:15021/healthz/ready": dial tcp 10.134.0.32:15021: connect: connection refused... 56m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Container created 56m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Container started 56m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 4 to 8 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Add eth0 [10.133.0.37/23] from ovn-kubernetes 56m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Container started 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Add eth0 [10.134.0.33/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-r5m84 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-whjkj 56m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Container created 56m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Add eth0 [10.132.0.56/23] from ovn-kubernetes 56m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Container created 56m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Container started 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Add eth0 [10.134.0.34/23] from ovn-kubernetes 56m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-dh7c4 56m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-klsmb 56m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Container started 56m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Container created 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Startup probe failed: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Startup probe failed: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Startup probe error: Get "http://10.132.0.56:15021/healthz/ready": dial tcp 10.132.0.56:15021: connect: connection refused... 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Startup probe failed: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Startup probe error: Get "http://10.134.0.33:15021/healthz/ready": dial tcp 10.134.0.33:15021: connect: connection refused... 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Startup probe error: Get "http://10.134.0.34:15021/healthz/ready": dial tcp 10.134.0.34:15021: connect: connection refused... 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 56m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Container started 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-v67pm 56m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Container created 56m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Add eth0 [10.133.0.39/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 8 to 9 56m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-l5qnm 56m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 56m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 56m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-44k4f 56m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 56m Normal AddedInterface pod/payload-processing-55fbbbf9dc-l5qnm Add eth0 [10.134.0.35/23] from ovn-kubernetes 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": dial tcp 10.133.0.39:15021: connect: connection refused... 56m Normal Pulling pod/payload-processing-55fbbbf9dc-l5qnm Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-44k4f Add eth0 [10.133.0.40/23] from ovn-kubernetes 56m Normal Pulling pod/payload-pre-processing-7b98bb678c-44k4f Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal Created pod/payload-pre-processing-7b98bb678c-44k4f Container created 56m Normal Pulled pod/payload-processing-55fbbbf9dc-l5qnm Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.139s (2.139s including waiting). Image size: 125353963 bytes. 56m Normal Created pod/payload-processing-55fbbbf9dc-l5qnm Container created 56m Normal Started pod/payload-pre-processing-7b98bb678c-44k4f Container started 56m Normal Started pod/payload-processing-55fbbbf9dc-l5qnm Container started 56m Normal Pulled pod/payload-pre-processing-7b98bb678c-44k4f Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.234s (2.234s including waiting). Image size: 125353963 bytes. 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Startup probe failed: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Startup probe error: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Container started 56m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c (combined from similar events): Created pod: maas-default-gateway-openshift-default-7f5667f86c-q757x 56m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 9 to 10 56m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Add eth0 [10.134.0.36/23] from ovn-kubernetes 56m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Container created 56m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused 56m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused... 55m Normal AddedLabel gateway/e2e-worker-w0-4a083d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-4a083d 55m Normal AddedLabel gateway/e2e-worker-w5-164db6 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-164db6 55m Normal AddedLabel gateway/e2e-worker-w6-43fac8 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-43fac8 55m Normal AddedLabel gateway/e2e-worker-w1-dfff59 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-dfff59 55m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 0 to 1 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled up replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 0 to 1 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled up replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 0 to 1 55m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled up replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled up replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 0 to 1 55m Normal AddedLabel gateway/e2e-worker-w2-6b0e0f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-6b0e0f 55m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled up replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 0 to 1 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf 55m Normal AddedLabel gateway/e2e-worker-w3-bd831a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-bd831a 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 55m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Add eth0 [10.133.0.42/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled up replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 0 to 1 55m Normal AddedLabel gateway/e2e-worker-w4-9f4dc9 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-9f4dc9 55m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Add eth0 [10.132.0.58/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws 55m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Add eth0 [10.132.0.60/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Add eth0 [10.134.0.38/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Add eth0 [10.132.0.59/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Add eth0 [10.133.0.43/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Add eth0 [10.134.0.37/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-43fac8 Scaled up replica set payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7 from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-43fac8 Scaled up replica set payload-processing-e2e-worker-w6-43fac8-86d989f9c from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-43fac8-86d989f9c Created pod: payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-43fac8-86d989f9c Error creating: pods "payload-processing-e2e-worker-w6-43fac8-86d989f9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-43fac8: serviceaccount "payload-processing-e2e-worker-w6-43fac8" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7 Created pod: payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7 Error creating: pods "payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-43fac8: serviceaccount "payload-processing-e2e-worker-w6-43fac8" not found 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5 Error creating: pods "payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-164db6: serviceaccount "payload-processing-e2e-worker-w5-164db6" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-4a083d-84c576c9b7 Created pod: payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm 55m Normal Started pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Container started 55m Normal Created pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Container created 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c Created pod: payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm 55m Normal Pulled pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Add eth0 [10.133.0.44/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-164db6 Scaled up replica set payload-processing-e2e-worker-w5-164db6-7848484d55 from 0 to 1 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-164db6-7848484d55 Created pod: payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-164db6 Scaled up replica set payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5 from 0 to 1 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Add eth0 [10.134.0.39/23] from ovn-kubernetes 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Container created 55m Normal Started pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Container started 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-164db6-7848484d55 Error creating: pods "payload-processing-e2e-worker-w5-164db6-7848484d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-164db6: serviceaccount "payload-processing-e2e-worker-w5-164db6" not found 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c Error creating: pods "payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-4a083d: serviceaccount "payload-processing-e2e-worker-w0-4a083d" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5 Created pod: payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-4a083d Scaled up replica set payload-processing-e2e-worker-w0-4a083d-84c576c9b7 from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-4a083d Scaled up replica set payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c from 0 to 1 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-4a083d-84c576c9b7 Error creating: pods "payload-processing-e2e-worker-w0-4a083d-84c576c9b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-4a083d: serviceaccount "payload-processing-e2e-worker-w0-4a083d" not found 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm Add eth0 [10.133.0.45/23] from ovn-kubernetes 55m Normal Started pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Container started 55m Normal Created pod/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm Container created 55m Normal Started pod/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm Container started 55m Normal AddedInterface pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Add eth0 [10.133.0.46/23] from ovn-kubernetes 55m Normal Pulled pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Pulled pod/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Container created 55m Normal AddedInterface pod/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm Add eth0 [10.134.0.40/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-dfff59 Scaled up replica set payload-processing-e2e-worker-w1-dfff59-688dc9df89 from 0 to 1 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm Container created 55m Normal Started pod/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm Container started 55m Normal Pulling pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-dfff59 Scaled up replica set payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c from 0 to 1 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Add eth0 [10.132.0.61/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5 Created pod: payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb 55m Normal Started pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Container started 55m Normal Created pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Container created 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-bd831a-544dc6b955 Created pod: payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Add eth0 [10.133.0.47/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c Error creating: pods "payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-dfff59: serviceaccount "payload-processing-e2e-worker-w1-dfff59" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c Created pod: payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-bd831a Scaled up replica set payload-processing-e2e-worker-w3-bd831a-544dc6b955 from 0 to 1 55m Normal Pulled pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Add eth0 [10.134.0.42/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-bd831a Scaled up replica set payload-pre-processing-e2e-worker-w3-bd831a-56899656f5 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5 Error creating: pods "payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-bd831a: serviceaccount "payload-processing-e2e-worker-w3-bd831a" not found 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-bd831a-544dc6b955 Error creating: pods "payload-processing-e2e-worker-w3-bd831a-544dc6b955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-bd831a: serviceaccount "payload-processing-e2e-worker-w3-bd831a" not found 55m Normal Pulled pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Add eth0 [10.133.0.48/23] from ovn-kubernetes 55m Normal Created pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Container created 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Add eth0 [10.134.0.43/23] from ovn-kubernetes 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-dfff59-688dc9df89 Created pod: payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-dfff59-688dc9df89 Error creating: pods "payload-processing-e2e-worker-w1-dfff59-688dc9df89-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-dfff59: serviceaccount "payload-processing-e2e-worker-w1-dfff59" not found 55m Normal Started pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Container started 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489 Error creating: pods "payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-9f4dc9: serviceaccount "payload-processing-e2e-worker-w4-9f4dc9" not found 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-9f4dc9 Scaled up replica set payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489 from 0 to 1 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899 Error creating: pods "payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-6b0e0f: serviceaccount "payload-processing-e2e-worker-w2-6b0e0f" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489 Created pod: payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx 55m Normal Started pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Container started 55m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf Error creating: pods "payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-6b0e0f: serviceaccount "payload-processing-e2e-worker-w2-6b0e0f" not found 55m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf Created pod: payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899 Created pod: payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p 55m Normal Started pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Container started 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Created pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Container created 55m Normal AddedInterface pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Add eth0 [10.133.0.49/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-6b0e0f Scaled up replica set payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899 from 0 to 1 55m Normal Created pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Container created 55m Normal Pulled pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Add eth0 [10.133.0.50/23] from ovn-kubernetes 55m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459 Error creating: pods "payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-9f4dc9: serviceaccount "payload-processing-e2e-worker-w4-9f4dc9" not found 55m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459 Created pod: payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s 55m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-6b0e0f Scaled up replica set payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf from 0 to 1 55m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-9f4dc9 Scaled up replica set payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459 from 0 to 1 55m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Add eth0 [10.133.0.51/23] from ovn-kubernetes 55m Normal Created pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Container created 55m Normal Created pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Container created 55m Normal Created pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Container created 55m Normal Started pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Container started 55m Normal Created pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Container created 55m Normal Started pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Container started 55m Normal AddedInterface pod/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx Add eth0 [10.132.0.63/23] from ovn-kubernetes 55m Normal Started pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Container started 55m Normal Pulled pod/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Started pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Container started 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 55m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.505s (2.505s including waiting). Image size: 125353963 bytes. 55m Normal Started pod/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx Container started 55m Normal Created pod/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx Container created 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": read tcp 10.133.0.2:40030->10.133.0.43:15021: read: connection reset by peer... 55m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Container started 55m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": read tcp 10.133.0.2:40030->10.133.0.43:15021: read: connection reset by peer 55m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Container created 55m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Container started 55m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Container created 55m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 55m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 55m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Container created 55m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 55m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Container started 55m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Container started 55m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Container created 55m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Container started 55m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Container created 55m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Container started 55m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Container started 55m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Container created 55m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Container created 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Startup probe failed: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Startup probe error: Get "http://10.132.0.58:15021/healthz/ready": dial tcp 10.132.0.58:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Startup probe failed: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Startup probe failed: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Startup probe error: Get "http://10.132.0.60:15021/healthz/ready": dial tcp 10.132.0.60:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Startup probe error: Get "http://10.132.0.59:15021/healthz/ready": dial tcp 10.132.0.59:15021: connect: connection refused... 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled up replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Container started 54m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Add eth0 [10.133.0.56/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Container created 54m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 54m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled up replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 1 to 2 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled up replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Add eth0 [10.134.0.49/23] from ovn-kubernetes 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Container created 54m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Add eth0 [10.134.0.50/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Add eth0 [10.133.0.58/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Container started 54m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Container created 54m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Container started 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled up replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 1 to 2 54m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Container created 54m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Container started 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr 54m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Startup probe error: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Startup probe failed: Get "http://10.133.0.58:15021/healthz/ready": dial tcp 10.133.0.58:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Startup probe error: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Startup probe error: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Startup probe failed: Get "http://10.134.0.50:15021/healthz/ready": dial tcp 10.134.0.50:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Startup probe failed: Get "http://10.134.0.49:15021/healthz/ready": dial tcp 10.134.0.49:15021: connect: connection refused 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs 54m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Container created 54m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Container started 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled up replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled up replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 2 to 3 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 1 to 2 54m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Add eth0 [10.134.0.51/23] from ovn-kubernetes 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Container created 54m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Add eth0 [10.132.0.73/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Container started 54m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Container created 54m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Container started 54m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Add eth0 [10.133.0.59/23] from ovn-kubernetes 54m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Startup probe failed: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Startup probe error: Get "http://10.133.0.59:15021/healthz/ready": dial tcp 10.133.0.59:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Startup probe failed: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Startup probe error: Get "http://10.132.0.73:15021/healthz/ready": dial tcp 10.132.0.73:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled up replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 1 to 2 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Add eth0 [10.133.0.60/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Container started 54m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Container created 54m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled up replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 2 to 3 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr 53m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Container started 53m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Add eth0 [10.134.0.55/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Container created 53m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Startup probe failed: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Startup probe error: Get "http://10.134.0.55:15021/healthz/ready": dial tcp 10.134.0.55:15021: connect: connection refused... 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled up replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 3 to 5 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled up replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled up replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv 53m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Add eth0 [10.134.0.58/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Container created 53m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Container started 53m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Add eth0 [10.133.0.72/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Container created 53m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Add eth0 [10.133.0.71/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Container started 53m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Container created 53m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Container started 53m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Container created 53m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Container started 53m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Add eth0 [10.132.0.80/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Container created 53m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Container started 53m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Add eth0 [10.132.0.79/23] from ovn-kubernetes 53m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Startup probe failed: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Startup probe error: Get "http://10.132.0.80:15021/healthz/ready": dial tcp 10.132.0.80:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled up replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 2 to 3 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Container started 53m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Container created 53m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Add eth0 [10.134.0.59/23] from ovn-kubernetes 53m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled up replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 2 to 4 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 3 to 4 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w 53m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Add eth0 [10.133.0.81/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Container started 53m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Container created 53m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Add eth0 [10.133.0.82/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Container created 53m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Container started 53m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Add eth0 [10.132.0.84/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp 53m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Add eth0 [10.132.0.85/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled up replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 3 to 6 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Container created 53m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Container started 53m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Add eth0 [10.134.0.62/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Container started 53m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Container started 53m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Container created 53m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Container started 53m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Add eth0 [10.134.0.61/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Container created 53m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Container created 52m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Startup probe failed: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Startup probe error: Get "http://10.132.0.85:15021/healthz/ready": dial tcp 10.132.0.85:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Startup probe error: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Startup probe failed: Get "http://10.134.0.62:15021/healthz/ready": dial tcp 10.134.0.62:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-27n7m 52m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 52m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 52m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Add eth0 [10.133.0.85/23] from ovn-kubernetes 52m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled up replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 3 to 6 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled up replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 3 to 6 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k 52m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Container created 52m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Add eth0 [10.134.0.64/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Container started 52m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Container created 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j 52m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Container started 52m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Add eth0 [10.132.0.89/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Add eth0 [10.133.0.87/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Container started 52m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Container created 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled up replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 3 to 5 52m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Container created 52m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Add eth0 [10.132.0.88/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Add eth0 [10.134.0.66/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Container created 52m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Container created 52m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Container started 52m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Container started 52m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z 52m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Container started 52m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Add eth0 [10.133.0.86/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Add eth0 [10.134.0.67/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Container created 52m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Container created 52m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Container started 52m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Add eth0 [10.132.0.87/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Container started 52m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Container created 52m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Container started 52m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 5.287s (5.287s including waiting). Image size: 185476477 bytes. 52m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Readiness probe error: HTTP probe failed with statuscode: 503... 52m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Readiness probe failed: HTTP probe failed with statuscode: 503 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": dial tcp 10.132.0.88:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Startup probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Startup probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled up replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 6 to 7 52m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw 52m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Container started 52m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Container created 52m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Add eth0 [10.134.0.68/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 4 to 6 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled up replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 5 to 10 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf (combined from similar events): Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Add eth0 [10.132.0.93/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Container created 52m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Container started 52m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Container started 52m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Container created 52m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Add eth0 [10.132.0.96/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Add eth0 [10.133.0.90/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Add eth0 [10.133.0.88/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Container created 52m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Container started 52m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Add eth0 [10.133.0.91/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Add eth0 [10.133.0.92/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Container created 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 (combined from similar events): Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Created pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 52m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Container started 52m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Container created 52m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Add eth0 [10.133.0.89/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Container started 52m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Container started 52m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Container created 52m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Add eth0 [10.132.0.94/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Add eth0 [10.134.0.69/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled up replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 5 to 8 52m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Container started 52m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Container created 52m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Add eth0 [10.132.0.95/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled up replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 6 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Created pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 52m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Container created 52m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Container started 52m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Container created 52m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Container started 52m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Container created 52m Normal Pulled pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Container created 52m Normal Started pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Container started 52m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Add eth0 [10.134.0.72/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Container created 52m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Add eth0 [10.134.0.73/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Container started 52m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Add eth0 [10.134.0.71/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Container started 52m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Container created 52m Normal Started pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Container started 52m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Container started 52m Normal Created pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Container created 52m Normal AddedInterface pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Add eth0 [10.134.0.70/23] from ovn-kubernetes 52m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Readiness probe failed: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Startup probe failed: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Startup probe failed: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Startup probe error: Get "http://10.132.0.94:15021/healthz/ready": dial tcp 10.132.0.94:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Startup probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Readiness probe error: Get "http://10.134.0.67:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m 51m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled up replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 4 to 8 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 6 to 8 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn 51m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 51m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled up replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 7 to 10 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 (combined from similar events): Created pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt 51m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Add eth0 [10.134.0.76/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Add eth0 [10.133.0.96/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Container created 51m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Container created 51m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Add eth0 [10.132.0.99/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Container started 51m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Container started 51m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Add eth0 [10.134.0.75/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Container started 51m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Container created 51m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Add eth0 [10.134.0.78/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Add eth0 [10.132.0.101/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Add eth0 [10.134.0.74/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Container created 51m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 51m Normal SuccessfulCreate replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc (combined from similar events): Created pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt 51m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Container created 51m Normal Created pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Container created 51m Normal Pulled pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Container started 51m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Container started 51m Normal AddedInterface pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Add eth0 [10.133.0.97/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Container created 51m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Container started 51m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Add eth0 [10.132.0.98/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Add eth0 [10.132.0.100/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Add eth0 [10.134.0.77/23] from ovn-kubernetes 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled up replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 8 to 10 51m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Container created 51m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Container started 51m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Container started 51m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Container created 51m Normal Created pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Container created 51m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Container created 51m Normal Started pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Container started 51m Normal Pulled pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Container started 51m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Container started 51m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Startup probe error: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Startup probe failed: Get "http://10.133.0.96:15021/healthz/ready": dial tcp 10.133.0.96:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Startup probe error: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Startup probe failed: Get "http://10.133.0.97:15021/healthz/ready": dial tcp 10.133.0.97:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Startup probe error: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Startup probe failed: Get "http://10.132.0.101:15021/healthz/ready": dial tcp 10.132.0.101:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Startup probe error: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Startup probe failed: Get "http://10.134.0.75:15021/healthz/ready": dial tcp 10.134.0.75:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled up replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 6 to 10 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b 51m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled up replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 8 to 10 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 (combined from similar events): Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Created pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj 51m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Container created 51m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Add eth0 [10.133.0.104/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Add eth0 [10.134.0.79/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Container started 51m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Container created 51m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Add eth0 [10.132.0.102/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c (combined from similar events): Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc 51m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Container created 51m Normal SuccessfulCreate replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Created pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq 51m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Container started 51m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Add eth0 [10.134.0.80/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Container created 51m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Container started 51m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Container started 51m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Container created 51m Normal AddedInterface pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Add eth0 [10.133.0.105/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Container started 51m Normal Pulled pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Pulled pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Add eth0 [10.133.0.106/23] from ovn-kubernetes 51m Normal Created pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Container created 51m Normal Started pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Container started 51m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Startup probe failed: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Startup probe error: Get "http://10.133.0.106:15021/healthz/ready": dial tcp 10.133.0.106:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Startup probe failed: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Startup probe error: Get "http://10.133.0.105:15021/healthz/ready": dial tcp 10.133.0.105:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Readiness probe error: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Readiness probe error: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Readiness probe failed: Get "http://10.132.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Readiness probe failed: Get "http://10.132.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Readiness probe failed: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Readiness probe failed: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Readiness probe error: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Readiness probe error: Get "http://10.134.0.55:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c (combined from similar events): Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr 50m Normal SuccessfulCreate replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Created pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps 50m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled up replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 8 to 10 50m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Container started 50m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Add eth0 [10.134.0.81/23] from ovn-kubernetes 50m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Container created 50m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Add eth0 [10.132.0.103/23] from ovn-kubernetes 50m Normal Started pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Container started 50m Normal Created pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Container created 50m Normal Pulled pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Readiness probe failed: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Readiness probe error: Get "http://10.133.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Readiness probe error: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Readiness probe failed: Get "http://10.133.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Readiness probe failed: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Readiness probe failed: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Readiness probe error: Get "http://10.133.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Readiness probe error: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Readiness probe failed: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Readiness probe failed: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Readiness probe failed: Get "http://10.134.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Readiness probe error: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Readiness probe error: Get "http://10.134.0.49:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Readiness probe failed: Get "http://10.132.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Readiness probe error: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Readiness probe failed: Get "http://10.134.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Readiness probe error: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Readiness probe failed: Get "http://10.132.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Readiness probe failed: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Readiness probe error: Get "http://10.133.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Readiness probe error: Get "http://10.133.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Readiness probe failed: Get "http://10.133.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Readiness probe error: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Readiness probe error: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Readiness probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Readiness probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Readiness probe error: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Readiness probe failed: Get "http://10.133.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Readiness probe error: Get "http://10.133.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Readiness probe error: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Readiness probe failed: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Readiness probe failed: Get "http://10.133.0.105:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Readiness probe failed: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Readiness probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Readiness probe error: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Readiness probe failed: Get "http://10.133.0.106:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Readiness probe failed: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Readiness probe error: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Readiness probe failed: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Readiness probe error: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Startup probe failed: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Readiness probe error: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Readiness probe failed: Get "http://10.132.0.101:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Readiness probe error: Get "http://10.134.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Startup probe error: Get "http://10.132.0.103:15021/healthz/ready": dial tcp 10.132.0.103:15021: connect: connection refused... 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Readiness probe error: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Readiness probe failed: Get "http://10.132.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Readiness probe error: Get "http://10.134.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Readiness probe error: Get "http://10.134.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal SuccessfulCreate replicaset/e2e-ait-8624bae9-openshift-default-5b6bc89bc5 Created pod: e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq 50m Normal ScalingReplicaSet deployment/e2e-ait-8624bae9-openshift-default Scaled up replica set e2e-ait-8624bae9-openshift-default-5b6bc89bc5 from 0 to 1 50m Normal AddedLabel gateway/e2e-ait-8624bae9 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8624bae9 50m Normal Created pod/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq Container created 50m Normal Started pod/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq Container started 50m Normal AddedInterface pod/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq Add eth0 [10.133.0.130/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp Container created 50m Normal SuccessfulCreate replicaset/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5 Created pod: e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp 50m Normal AddedInterface pod/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp Add eth0 [10.132.0.106/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp Container started 50m Normal AddedLabel gateway/e2e-migrate-3664624a-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-3664624a-gw 50m Normal ScalingReplicaSet deployment/e2e-migrate-3664624a-gw-openshift-default Scaled up replica set e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67 Error creating: pods "payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8624bae9: serviceaccount "payload-processing-e2e-ait-8624bae9" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67 Created pod: payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-8624bae9-548c6d49df Created pod: payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s 50m Warning FailedCreate replicaset/payload-processing-e2e-ait-8624bae9-548c6d49df Error creating: pods "payload-processing-e2e-ait-8624bae9-548c6d49df-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8624bae9: serviceaccount "payload-processing-e2e-ait-8624bae9" not found 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-8624bae9 Scaled up replica set payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-8624bae9 Scaled up replica set payload-processing-e2e-ait-8624bae9-548c6d49df from 0 to 1 50m Normal AddedInterface pod/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s Add eth0 [10.133.0.131/23] from ovn-kubernetes 50m Normal Created pod/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf Container created 50m Normal Pulled pod/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf Container started 50m Normal Started pod/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf Add eth0 [10.134.0.82/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s Container created 50m Warning FailedCreate replicaset/payload-processing-e2e-migrate-3664624a-68966c9b49 Error creating: pods "payload-processing-e2e-migrate-3664624a-68966c9b49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-3664624a: serviceaccount "payload-processing-e2e-migrate-3664624a" not found 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-3664624a-64bd765999 Error creating: pods "payload-pre-processing-e2e-migrate-3664624a-64bd765999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-3664624a: serviceaccount "payload-processing-e2e-migrate-3664624a" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-3664624a-64bd765999 Created pod: payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-3664624a Scaled up replica set payload-pre-processing-e2e-migrate-3664624a-64bd765999 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-3664624a Scaled up replica set payload-processing-e2e-migrate-3664624a-68966c9b49 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-3664624a-68966c9b49 Created pod: payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg 50m Normal Created pod/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg Container created 50m Normal AddedInterface pod/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg Add eth0 [10.134.0.84/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg Container started 50m Normal Created pod/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf Container created 50m Normal Pulled pod/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf Add eth0 [10.134.0.83/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf Container started 50m Normal Pulled pod/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm Add eth0 [10.133.0.132/23] from ovn-kubernetes 50m Normal Created pod/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm Container created 50m Normal SuccessfulCreate replicaset/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9 Created pod: e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm 50m Normal ScalingReplicaSet deployment/e2e-ait-ec6894c9-openshift-default Scaled up replica set e2e-ait-ec6894c9-openshift-default-6f8c69ffc9 from 0 to 1 50m Normal AddedLabel gateway/e2e-ait-ec6894c9 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-ec6894c9 50m Normal Started pod/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm Container started 50m Normal SuccessfulCreate replicaset/e2e-mt-47e1515a-openshift-default-69dc94c849 Created pod: e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 50m Normal ScalingReplicaSet deployment/e2e-mt-47e1515a-openshift-default Scaled up replica set e2e-mt-47e1515a-openshift-default-69dc94c849 from 0 to 1 50m Normal AddedLabel gateway/e2e-mt-47e1515a Added label istio.io/rev=openshift-gateway to gateway e2e-mt-47e1515a 50m Normal AddedInterface pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Add eth0 [10.132.0.107/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-ec6894c9 Scaled up replica set payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9 Error creating: pods "payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ec6894c9: serviceaccount "payload-processing-e2e-ait-ec6894c9" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9 Created pod: payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw 50m Warning FailedCreate replicaset/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f Error creating: pods "payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-ec6894c9: serviceaccount "payload-processing-e2e-ait-ec6894c9" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f Created pod: payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-ec6894c9 Scaled up replica set payload-processing-e2e-ait-ec6894c9-77f7f5dc8f from 0 to 1 50m Normal AddedInterface pod/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs Add eth0 [10.134.0.87/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw Add eth0 [10.134.0.86/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs Container started 49m Normal Created pod/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs Container created 49m Normal Created pod/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw Container created 49m Normal Started pod/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw Container started 49m Warning FailedCreate replicaset/payload-processing-e2e-mt-47e1515a-5df486c565 Error creating: pods "payload-processing-e2e-mt-47e1515a-5df486c565-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-47e1515a: serviceaccount "payload-processing-e2e-mt-47e1515a" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-47e1515a Scaled up replica set payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-47e1515a-5df486c565 Created pod: payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c Created pod: payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-47e1515a Scaled up replica set payload-processing-e2e-mt-47e1515a-5df486c565 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c Error creating: pods "payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-47e1515a: serviceaccount "payload-processing-e2e-mt-47e1515a" not found 49m Normal Started pod/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf Container started 49m Normal AddedInterface pod/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz Add eth0 [10.133.0.133/23] from ovn-kubernetes 49m Normal Created pod/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf Add eth0 [10.133.0.134/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz Container started 49m Normal Created pod/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz Container created 49m Normal Pulled pod/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Readiness probe error: Get "http://10.132.0.95:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Readiness probe error: Get "http://10.132.0.94:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8624bae9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8624bae9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-3664624a-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-3664624a-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-2qs8x 49m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 49m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 4 49m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-rtv2k 49m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Add eth0 [10.132.0.110/23] from ovn-kubernetes 49m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Container created 49m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Add eth0 [10.133.0.137/23] from ovn-kubernetes 49m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 49m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Container started 49m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Readiness probe error: HTTP probe failed with statuscode: 503... 49m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Readiness probe failed: HTTP probe failed with statuscode: 503 49m Normal Killing pod/payload-pre-processing-e2e-ait-8624bae9-f4ffc7b67-pcczf Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-ait-8624bae9-548c6d49df-p5h5s Stopping container payload-processing 49m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Container created 49m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.945s (3.945s including waiting). Image size: 185476477 bytes. 49m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Container started 49m Normal Killing pod/payload-pre-processing-e2e-migrate-3664624a-64bd765999-zjrqf Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-migrate-3664624a-68966c9b49-8hqvg Stopping container payload-processing 49m Normal AddedLabel gateway/e2e-ait-474cab5e Added label istio.io/rev=openshift-gateway to gateway e2e-ait-474cab5e 49m Normal Killing pod/e2e-ait-8624bae9-openshift-default-5b6bc89bc5-r29nq Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/e2e-ait-474cab5e-openshift-default Scaled up replica set e2e-ait-474cab5e-openshift-default-5c5fff6655 from 0 to 1 49m Normal SuccessfulCreate replicaset/e2e-ait-474cab5e-openshift-default-5c5fff6655 Created pod: e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 49m Normal Started pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Container started 49m Normal AddedInterface pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Add eth0 [10.134.0.89/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Killing pod/payload-pre-processing-e2e-ait-ec6894c9-57bc8d4f9-2dkbw Stopping container payload-pre-processing 49m Normal Killing pod/payload-processing-e2e-ait-ec6894c9-77f7f5dc8f-hstfs Stopping container payload-processing 49m Normal Created pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Container created 49m Normal Killing pod/e2e-migrate-3664624a-gw-openshift-default-7ff9b75cb5-dgsdp Stopping container istio-proxy 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-ec6894c9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-ec6894c9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal AddedLabel gateway/e2e-derive-9fd01d20 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-9fd01d20 49m Normal ScalingReplicaSet deployment/e2e-derive-9fd01d20-openshift-default Scaled up replica set e2e-derive-9fd01d20-openshift-default-744446d849 from 0 to 1 49m Normal SuccessfulCreate replicaset/e2e-derive-9fd01d20-openshift-default-744446d849 Created pod: e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-474cab5e Scaled up replica set payload-pre-processing-e2e-ait-474cab5e-79b66887b7 from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-474cab5e Scaled up replica set payload-processing-e2e-ait-474cab5e-85488d9c5c from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-ait-474cab5e-85488d9c5c Error creating: pods "payload-processing-e2e-ait-474cab5e-85488d9c5c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-474cab5e: serviceaccount "payload-processing-e2e-ait-474cab5e" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-474cab5e-79b66887b7 Error creating: pods "payload-pre-processing-e2e-ait-474cab5e-79b66887b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-474cab5e: serviceaccount "payload-processing-e2e-ait-474cab5e" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-474cab5e-79b66887b7 Created pod: payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-474cab5e-85488d9c5c Created pod: payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b 49m Normal Created pod/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j Add eth0 [10.134.0.91/23] from ovn-kubernetes 49m Normal Pulled pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b Container created 49m Normal AddedInterface pod/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b Add eth0 [10.134.0.92/23] from ovn-kubernetes 49m Normal AddedInterface pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Add eth0 [10.134.0.90/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Container created 49m Normal Started pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Container started 49m Normal Started pod/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b Container started 49m Normal Started pod/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j Container started 49m Normal Pulled pod/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Killing pod/e2e-ait-ec6894c9-openshift-default-6f8c69ffc9-sjgjm Stopping container istio-proxy 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-9fd01d20 Scaled up replica set payload-pre-processing-e2e-derive-9fd01d20-755c777847 from 0 to 1 49m Warning FailedCreate replicaset/payload-processing-e2e-derive-9fd01d20-7b479cbc8c Error creating: pods "payload-processing-e2e-derive-9fd01d20-7b479cbc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-9fd01d20: serviceaccount "payload-processing-e2e-derive-9fd01d20" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-9fd01d20-7b479cbc8c Created pod: payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-9fd01d20 Scaled up replica set payload-processing-e2e-derive-9fd01d20-7b479cbc8c from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-9fd01d20-755c777847 Created pod: payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-9fd01d20-755c777847 Error creating: pods "payload-pre-processing-e2e-derive-9fd01d20-755c777847-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-9fd01d20: serviceaccount "payload-processing-e2e-derive-9fd01d20" not found 49m Normal Pulled pod/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Pulled pod/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 Container created 49m Normal Started pod/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx Container started 49m Normal Started pod/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 Container started 49m Normal Created pod/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx Container created 49m Normal AddedInterface pod/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 Add eth0 [10.134.0.94/23] from ovn-kubernetes 49m Normal AddedInterface pod/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx Add eth0 [10.134.0.93/23] from ovn-kubernetes 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-474cab5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-474cab5e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Pulled pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Container created 49m Normal Started pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Container started 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-9fd01d20-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-9fd01d20-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-47e1515a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-47e1515a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Startup probe error: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Startup probe failed: Get "http://10.132.0.107:15021/healthz/ready": dial tcp 10.132.0.107:15021: connect: connection refused 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-474cab5e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-474cab5e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-9fd01d20-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-9fd01d20-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal Killing pod/payload-pre-processing-e2e-derive-9fd01d20-755c777847-lqtlx Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-derive-9fd01d20-7b479cbc8c-mt2d2 Stopping container payload-processing 48m Normal Killing pod/payload-processing-e2e-ait-474cab5e-85488d9c5c-btq6b Stopping container payload-processing 48m Normal Killing pod/payload-pre-processing-e2e-ait-474cab5e-79b66887b7-brn4j Stopping container payload-pre-processing 48m Normal Killing pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Stopping container istio-proxy 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-383fde-openshift-default Scaled up replica set e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 from 0 to 1 48m Normal AddedLabel gateway/e2e-shared-a-w5-383fde Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-383fde 48m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 Created pod: e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 48m Normal AddedInterface pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Add eth0 [10.133.0.141/23] from ovn-kubernetes 48m Normal Killing pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Stopping container istio-proxy 48m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f Created pod: e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5e2d03-openshift-default Scaled up replica set e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f from 0 to 1 48m Normal AddedLabel gateway/e2e-shared-a-w0-5e2d03 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-5e2d03 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655 Created pod: payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-383fde Scaled up replica set payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-383fde: serviceaccount "payload-processing-e2e-shared-a-w5-383fde" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-383fde-8486d66857 Created pod: payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-383fde-8486d66857 Error creating: pods "payload-processing-e2e-shared-a-w5-383fde-8486d66857-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-383fde: serviceaccount "payload-processing-e2e-shared-a-w5-383fde" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-383fde Scaled up replica set payload-processing-e2e-shared-a-w5-383fde-8486d66857 from 0 to 1 48m Normal AddedInterface pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Add eth0 [10.134.0.97/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Container created 48m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Add eth0 [10.134.0.98/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Container started 48m Warning ProbeError pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Normal Created pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Container created 48m Normal Started pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Container started 48m Normal Pulled pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Add eth0 [10.134.0.99/23] from ovn-kubernetes 48m Warning Unhealthy pod/e2e-ait-474cab5e-openshift-default-5c5fff6655-dpt54 Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-5e2d03 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc Error creating: pods "payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-5e2d03: serviceaccount "payload-processing-e2e-shared-a-w0-5e2d03" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc Created pod: payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-5e2d03 Scaled up replica set payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9 Error creating: pods "payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-5e2d03: serviceaccount "payload-processing-e2e-shared-a-w0-5e2d03" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9 Created pod: payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Add eth0 [10.134.0.101/23] from ovn-kubernetes 48m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Container created 48m Warning ProbeError pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Readiness probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-derive-9fd01d20-openshift-default-744446d849-cstp9 Readiness probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Container started 48m Normal Started pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Container started 48m Normal Created pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Container created 48m Normal Pulled pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Add eth0 [10.134.0.100/23] from ovn-kubernetes 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-5e2d03-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-5e2d03-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-47e1515a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-47e1515a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedLabel gateway/e2e-shared-b-w5-a9a967 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-a9a967 48m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled up replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m 48m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Add eth0 [10.134.0.104/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-c8d121-openshift-default Scaled up replica set e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c from 0 to 1 48m Normal AddedLabel gateway/e2e-shared-b-w0-c8d121 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-c8d121 48m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c Created pod: e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv 48m Normal AddedInterface pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Add eth0 [10.132.0.111/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-a9a967 Scaled up replica set payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-a9a967: serviceaccount "payload-processing-e2e-shared-b-w5-a9a967" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c Error creating: pods "payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-a9a967: serviceaccount "payload-processing-e2e-shared-b-w5-a9a967" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-a9a967 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67 Created pod: payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj 48m Normal Pulled pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Container created 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c Created pod: payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 48m Normal Started pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Add eth0 [10.134.0.105/23] from ovn-kubernetes 48m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Add eth0 [10.133.0.142/23] from ovn-kubernetes 48m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Container started 48m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Container created 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85 Created pod: payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559 Error creating: pods "payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-c8d121: serviceaccount "payload-processing-e2e-shared-b-w0-c8d121" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559 Created pod: payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-c8d121 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-c8d121 Scaled up replica set payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85 Error creating: pods "payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-c8d121: serviceaccount "payload-processing-e2e-shared-b-w0-c8d121" not found 48m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Container started 48m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Container created 48m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Add eth0 [10.132.0.112/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Add eth0 [10.134.0.106/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Container created 48m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Container started 47m Normal Killing pod/payload-processing-e2e-mt-47e1515a-5df486c565-cqhgz Stopping container payload-processing 47m Normal ScalingReplicaSet deployment/e2e-mt-76063d10-openshift-default Scaled up replica set e2e-mt-76063d10-openshift-default-54bbfd48b9 from 0 to 1 47m Normal Killing pod/payload-pre-processing-e2e-mt-47e1515a-5cf9b7c76c-2wlwf Stopping container payload-pre-processing 47m Normal AddedLabel gateway/e2e-mt-76063d10 Added label istio.io/rev=openshift-gateway to gateway e2e-mt-76063d10 47m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m 47m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Add eth0 [10.133.0.144/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-76063d10 Scaled up replica set payload-pre-processing-e2e-mt-76063d10-78f95fdf8c from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-mt-76063d10-f66f6f4bf Error creating: pods "payload-processing-e2e-mt-76063d10-f66f6f4bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-76063d10: serviceaccount "payload-processing-e2e-mt-76063d10" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-76063d10-f66f6f4bf Created pod: payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-76063d10 Scaled up replica set payload-processing-e2e-mt-76063d10-f66f6f4bf from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c Created pod: payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c Error creating: pods "payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-76063d10: serviceaccount "payload-processing-e2e-mt-76063d10" not found 47m Normal Pulled pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Add eth0 [10.134.0.107/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Add eth0 [10.134.0.108/23] from ovn-kubernetes 47m Normal Started pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Container started 47m Normal Created pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Container created 47m Normal Pulled pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Container started 47m Normal Created pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Container created 47m Warning Unhealthy pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Readiness probe failed: Get "http://10.133.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning ProbeError pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Readiness probe error: Get "http://10.133.0.141:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Normal Started pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Container started 47m Normal Pulled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Container created 47m Warning ProbeError pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Readiness probe error: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Readiness probe failed: Get "http://10.134.0.97:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Killing pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Stopping container istio-proxy 47m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 47m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd 47m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Warning ProbeError pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Add eth0 [10.132.0.116/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-mt-47e1515a-openshift-default-69dc94c849-7vsn4 Readiness probe failed: HTTP probe failed with statuscode: 503 47m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled up replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 1 to 4 47m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv 47m Normal Started pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Container started 47m Normal Pulled pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Container created 47m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Container started 47m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Container created 47m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Container started 47m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Container created 47m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Add eth0 [10.133.0.145/23] from ovn-kubernetes 47m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Add eth0 [10.134.0.112/23] from ovn-kubernetes 47m Normal Created pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Container created 47m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Container started 47m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Container started 47m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Container created 47m Warning Unhealthy pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Startup probe failed: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused 47m Warning ProbeError pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Startup probe error: Get "http://10.133.0.141:15021/healthz/ready": dial tcp 10.133.0.141:15021: connect: connection refused... 47m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n 47m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled up replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 0 to 1 47m Normal AddedLabel gateway/e2e-shared-a-w3-f47890 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-f47890 47m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Add eth0 [10.133.0.146/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Startup probe failed: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused 47m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Startup probe error: Get "http://10.132.0.116:15021/healthz/ready": dial tcp 10.132.0.116:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-f47890 Scaled up replica set payload-processing-e2e-shared-a-w3-f47890-b7b766d4c from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c Created pod: payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c Error creating: pods "payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-f47890: serviceaccount "payload-processing-e2e-shared-a-w3-f47890" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59 Error creating: pods "payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-f47890: serviceaccount "payload-processing-e2e-shared-a-w3-f47890" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59 Created pod: payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-f47890 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59 from 0 to 1 47m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Add eth0 [10.132.0.117/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Container started 47m Normal Created pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Container created 47m Normal Pulled pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Startup probe failed: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused 47m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Startup probe failed: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Add eth0 [10.134.0.114/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Startup probe error: Get "http://10.134.0.104:15021/healthz/ready": dial tcp 10.134.0.104:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Startup probe error: Get "http://10.134.0.112:15021/healthz/ready": dial tcp 10.134.0.112:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Startup probe failed: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused 47m Warning ProbeError pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Startup probe error: Get "http://10.134.0.97:15021/healthz/ready": dial tcp 10.134.0.97:15021: connect: connection refused... 47m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Container created 47m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Container started 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal SuccessfulCreate replicaset/e2e-api-a-61bb54-openshift-default-84944ff49c Created pod: e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt 47m Normal ScalingReplicaSet deployment/e2e-api-a-61bb54-openshift-default Scaled up replica set e2e-api-a-61bb54-openshift-default-84944ff49c from 0 to 1 47m Normal AddedLabel gateway/e2e-api-a-61bb54 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-61bb54 47m Normal AddedInterface pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Add eth0 [10.133.0.148/23] from ovn-kubernetes 47m Warning FailedCreate replicaset/payload-processing-e2e-api-a-61bb54-579844cd75 Error creating: pods "payload-processing-e2e-api-a-61bb54-579844cd75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-61bb54: serviceaccount "payload-processing-e2e-api-a-61bb54" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-61bb54-84479f79f5 Error creating: pods "payload-pre-processing-e2e-api-a-61bb54-84479f79f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-61bb54: serviceaccount "payload-processing-e2e-api-a-61bb54" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-61bb54-84479f79f5 Created pod: payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-61bb54 Scaled up replica set payload-pre-processing-e2e-api-a-61bb54-84479f79f5 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-61bb54 Scaled up replica set payload-processing-e2e-api-a-61bb54-579844cd75 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-61bb54-579844cd75 Created pod: payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv 47m Normal Pulled pod/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv Add eth0 [10.133.0.149/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd Add eth0 [10.134.0.116/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd Container created 47m Normal Created pod/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv Container created 47m Normal Started pod/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv Container started 47m Normal Started pod/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd Container started 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedLabel gateway/e2e-shared-b-w3-734859 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-734859 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f Created pod: e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-734859-openshift-default Scaled up replica set e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f from 0 to 1 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-383fde-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-383fde-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal AddedInterface pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Add eth0 [10.133.0.151/23] from ovn-kubernetes 46m Normal AddedLabel gateway/e2e-shared-a-w4-9a482b Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-9a482b 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9a482b-openshift-default Scaled up replica set e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz 46m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Add eth0 [10.132.0.119/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-734859 Scaled up replica set payload-processing-e2e-shared-b-w3-734859-5fc5fb544f from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f Error creating: pods "payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-734859: serviceaccount "payload-processing-e2e-shared-b-w3-734859" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f Created pod: payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787 Created pod: payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-734859 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-734859-589cbc787 from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-734859: serviceaccount "payload-processing-e2e-shared-b-w3-734859" not found 46m Normal Started pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Add eth0 [10.134.0.119/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Container created 46m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Container started 46m Normal Created pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Container created 46m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Add eth0 [10.134.0.118/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f Error creating: pods "payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9a482b: serviceaccount "payload-processing-e2e-shared-a-w4-9a482b" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-9a482b Scaled up replica set payload-processing-e2e-shared-a-w4-9a482b-86b559b58f from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995 Created pod: payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-9a482b Scaled up replica set payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995 from 0 to 1 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-9a482b: serviceaccount "payload-processing-e2e-shared-a-w4-9a482b" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f Created pod: payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 46m Normal Created pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Container created 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Add eth0 [10.134.0.121/23] from ovn-kubernetes 46m Normal Pulled pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Container started 46m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Add eth0 [10.134.0.120/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Container created 46m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Container started 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default New size: 5; reason: All metrics below target 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Container created 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Container started 46m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Container created 46m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Readiness probe failed: Get "http://10.133.0.144:15021/healthz/ready": read tcp 10.133.0.2:57924->10.133.0.144:15021: read: connection reset by peer 46m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Container started 46m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Readiness probe error: Get "http://10.133.0.144:15021/healthz/ready": read tcp 10.133.0.2:57924->10.133.0.144:15021: read: connection reset by peer... 46m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Container created 46m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Add eth0 [10.133.0.152/23] from ovn-kubernetes 46m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Container started 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-c8d121-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning ProbeError pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-c8d121-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 46m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Startup probe error: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused... 46m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Startup probe failed: Get "http://10.133.0.144:15021/healthz/ready": dial tcp 10.133.0.144:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-5e2d03-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-5e2d03-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-76063d10-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-76063d10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulCreate replicaset/e2e-api-b-59d3a7-openshift-default-dcfc79fcc Created pod: e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h 46m Normal AddedLabel gateway/e2e-api-b-59d3a7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-59d3a7 46m Normal ScalingReplicaSet deployment/e2e-api-b-59d3a7-openshift-default Scaled up replica set e2e-api-b-59d3a7-openshift-default-dcfc79fcc from 0 to 1 46m Normal AddedInterface pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Add eth0 [10.134.0.122/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-59d3a7 Scaled up replica set payload-processing-e2e-api-b-59d3a7-7d8d884767 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-api-b-59d3a7-7d8d884767 Error creating: pods "payload-processing-e2e-api-b-59d3a7-7d8d884767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-59d3a7: serviceaccount "payload-processing-e2e-api-b-59d3a7" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-59d3a7-7d8d884767 Created pod: payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7 Error creating: pods "payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-59d3a7: serviceaccount "payload-processing-e2e-api-b-59d3a7" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7 Created pod: payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-59d3a7 Scaled up replica set payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7 from 0 to 1 46m Normal Started pod/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs Container started 46m Normal Created pod/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs Container created 46m Normal AddedInterface pod/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft Add eth0 [10.134.0.123/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft Container created 46m Normal Started pod/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs Add eth0 [10.134.0.124/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-76063d10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-76063d10-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-f01de5-openshift-default Scaled up replica set e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f Created pod: e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 46m Normal AddedLabel gateway/e2e-shared-b-w4-f01de5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-f01de5 46m Normal AddedInterface pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Add eth0 [10.133.0.153/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-f01de5 Scaled up replica set payload-processing-e2e-shared-b-w4-f01de5-65884f55d5 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5 Error creating: pods "payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-f01de5: serviceaccount "payload-processing-e2e-shared-b-w4-f01de5" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87 Created pod: payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-f01de5 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5 Created pod: payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-c8d121-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-c8d121-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87 Error creating: pods "payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-f01de5: serviceaccount "payload-processing-e2e-shared-b-w4-f01de5" not found 46m Normal Pulled pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Add eth0 [10.134.0.125/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Container created 46m Normal Started pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Container started 46m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Add eth0 [10.133.0.154/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Container created 46m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Container started 45m Normal Pulled pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled up replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 1 to 4 45m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 45m Normal Pulled pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll 45m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Container started 45m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Add eth0 [10.133.0.155/23] from ovn-kubernetes 45m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Container started 45m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Add eth0 [10.132.0.124/23] from ovn-kubernetes 45m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Container created 45m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Container created 45m Normal Created pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Container created 45m Normal Created pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Container created 45m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Container created 45m Normal Started pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Container started 45m Normal Started pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Container started 45m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Container started 45m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Container started 45m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Container created 45m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Add eth0 [10.132.0.123/23] from ovn-kubernetes 45m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Startup probe failed: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Startup probe error: Get "http://10.132.0.124:15021/healthz/ready": dial tcp 10.132.0.124:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-f01de5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Startup probe error: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Startup probe failed: Get "http://10.133.0.148:15021/healthz/ready": dial tcp 10.133.0.148:15021: connect: connection refused 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-f01de5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Startup probe failed: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Startup probe error: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-734859-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-734859-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-61bb54-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-61bb54-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-pre-processing-e2e-api-a-61bb54-84479f79f5-rc5cd Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-api-a-61bb54-579844cd75-zwdpv Stopping container payload-processing 45m Normal ScalingReplicaSet deployment/e2e-api-a-5c6b75-openshift-default Scaled up replica set e2e-api-a-5c6b75-openshift-default-86f46b878f from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-api-a-5c6b75-openshift-default-86f46b878f Created pod: e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p 45m Normal AddedLabel gateway/e2e-api-a-5c6b75 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-5c6b75 45m Normal AddedInterface pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Add eth0 [10.133.0.158/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Container created 45m Normal Started pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Container started 45m Warning FailedCreate replicaset/payload-processing-e2e-api-a-5c6b75-58cdb8f699 Error creating: pods "payload-processing-e2e-api-a-5c6b75-58cdb8f699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-5c6b75: serviceaccount "payload-processing-e2e-api-a-5c6b75" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787 Error creating: pods "payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-5c6b75: serviceaccount "payload-processing-e2e-api-a-5c6b75" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-5c6b75 Scaled up replica set payload-processing-e2e-api-a-5c6b75-58cdb8f699 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-5c6b75-58cdb8f699 Created pod: payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-5c6b75 Scaled up replica set payload-pre-processing-e2e-api-a-5c6b75-9f88c8787 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787 Created pod: payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j 45m Normal Created pod/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 Container created 45m Normal AddedInterface pod/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 Add eth0 [10.134.0.126/23] from ovn-kubernetes 45m Normal Started pod/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j Add eth0 [10.134.0.127/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j Container created 45m Normal Pulled pod/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j Container started 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-61bb54-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-61bb54-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Pulled pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Container created 45m Normal Started pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Container started 45m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 5 45m Normal SuccessfulCreate replicaset/e2e-api-a-5c6b75-openshift-default-86f46b878f Created pod: e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b 45m Normal Started pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Container started 45m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-fv7j6 45m Warning Unhealthy pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal Pulled pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Killing pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-cjk5p Stopping container istio-proxy 45m Normal Created pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Container created 45m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 5; reason: cpu resource utilization (percentage of request) above target 45m Normal AddedInterface pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Add eth0 [10.132.0.126/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-59d3a7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-59d3a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning ProbeError pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused 45m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Container started 45m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 45m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Container created 45m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Add eth0 [10.133.0.160/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled up replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 4 to 5 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default New size: 5; reason: All metrics below target 45m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-734859-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-734859-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Startup probe error: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused... 45m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning Unhealthy pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Startup probe failed: Get "http://10.134.0.122:15021/healthz/ready": dial tcp 10.134.0.122:15021: connect: connection refused 45m Normal ScalingReplicaSet deployment/e2e-api-a-61bb54-openshift-default Scaled up replica set e2e-api-a-61bb54-openshift-default-84944ff49c from 0 to 1 45m Normal Killing pod/e2e-api-a-61bb54-openshift-default-84944ff49c-p9zmt Stopping container istio-proxy 45m Warning FailedCreate replicaset/e2e-api-a-61bb54-openshift-default-84944ff49c Error creating: pods "e2e-api-a-61bb54-openshift-default-84944ff49c-" is forbidden: error looking up service account openshift-ingress/e2e-api-a-61bb54-openshift-default: serviceaccount "e2e-api-a-61bb54-openshift-default" not found 45m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Add eth0 [10.133.0.163/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Container started 45m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Container created 45m Normal Killing pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Stopping container payload-pre-processing 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-59d3a7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-59d3a7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Killing pod/payload-processing-e2e-api-b-59d3a7-7d8d884767-mc7ft Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-api-b-59d3a7-5c65656bb7-gbhbs Stopping container payload-pre-processing 44m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 44m Normal ScalingReplicaSet deployment/e2e-api-b-80c4c8-openshift-default Scaled up replica set e2e-api-b-80c4c8-openshift-default-78ff9c847 from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-api-b-80c4c8-openshift-default-78ff9c847 Created pod: e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph 44m Normal AddedLabel gateway/e2e-api-b-80c4c8 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-80c4c8 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled down replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 10 to 9 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 9; reason: All metrics below target 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-f01de5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-f01de5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Stopping container istio-proxy 44m Warning ProbeError pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-kv5zt Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning Unhealthy pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-rsvc9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 44m Warning Unhealthy pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-9p2wc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 44m Normal AddedInterface pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Add eth0 [10.134.0.130/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-processing-e2e-api-b-80c4c8-75974fbc49 Error creating: pods "payload-processing-e2e-api-b-80c4c8-75974fbc49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-80c4c8: serviceaccount "payload-processing-e2e-api-b-80c4c8" not found 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-80c4c8-5779b74587 Error creating: pods "payload-pre-processing-e2e-api-b-80c4c8-5779b74587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-80c4c8: serviceaccount "payload-processing-e2e-api-b-80c4c8" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-80c4c8 Scaled up replica set payload-pre-processing-e2e-api-b-80c4c8-5779b74587 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-80c4c8-5779b74587 Created pod: payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-80c4c8-75974fbc49 Created pod: payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-80c4c8 Scaled up replica set payload-processing-e2e-api-b-80c4c8-75974fbc49 from 0 to 1 44m Normal Started pod/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg Add eth0 [10.134.0.132/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh Container started 44m Normal AddedInterface pod/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh Add eth0 [10.134.0.131/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg Container created 44m Normal Pulled pod/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh Container created 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled down replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 10 to 9 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 44m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-gsxq6 Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 9; reason: All metrics below target 44m Warning Unhealthy pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Readiness probe failed: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Warning ProbeError pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Readiness probe error: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Normal Pulled pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Container created 44m Normal Started pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Container started 44m Normal Killing pod/e2e-api-b-59d3a7-openshift-default-dcfc79fcc-4tr8h Stopping container istio-proxy 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-76063d10-f66f6f4bf Created pod: payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz 44m Warning FailedCreate replicaset/payload-processing-e2e-mt-76063d10-f66f6f4bf Error creating: pods "payload-processing-e2e-mt-76063d10-f66f6f4bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-76063d10: serviceaccount "payload-processing-e2e-mt-76063d10" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-76063d10 Scaled up replica set payload-pre-processing-e2e-mt-76063d10-78f95fdf8c from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-76063d10 Scaled up replica set payload-processing-e2e-mt-76063d10-f66f6f4bf from 0 to 1 44m Normal AddedLabel gateway/e2e-inf-a-f5dd93 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f5dd93 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c Error creating: pods "payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-76063d10: serviceaccount "payload-processing-e2e-mt-76063d10" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c Created pod: payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt 44m Normal ScalingReplicaSet deployment/e2e-inf-a-f5dd93-openshift-default Scaled up replica set e2e-inf-a-f5dd93-openshift-default-86c6c56f5c from 0 to 1 44m Normal SuccessfulCreate replicaset/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c Created pod: e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 44m Normal Started pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz Container started 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-5c6b75-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedInterface pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz Add eth0 [10.134.0.134/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 44m Normal Pulled pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz Container created 44m Normal Created pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt Add eth0 [10.134.0.133/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt Container started 44m Normal Pulled pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Add eth0 [10.134.0.135/23] from ovn-kubernetes 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-5c6b75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 9; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled down replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 10 to 9 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-qvf4f 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 9; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled down replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 10 to 9 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 9; reason: All metrics below target 44m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled down replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 10 to 9 44m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-pnlqn Stopping container istio-proxy 44m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-xdtnt Stopping container istio-proxy 44m Normal SuccessfulCreate replicaset/e2e-api-a-022f62-openshift-default-c7698fd8f Created pod: e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p 44m Normal AddedLabel gateway/e2e-api-a-022f62 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-022f62 44m Normal ScalingReplicaSet deployment/e2e-api-a-022f62-openshift-default Scaled up replica set e2e-api-a-022f62-openshift-default-c7698fd8f from 0 to 1 44m Normal AddedInterface pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Add eth0 [10.134.0.136/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f5dd93-85fd67b947 Created pod: payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5 Error creating: pods "payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f5dd93: serviceaccount "payload-processing-e2e-inf-a-f5dd93" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f5dd93 Scaled up replica set payload-processing-e2e-inf-a-f5dd93-85fd67b947 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f5dd93 Scaled up replica set payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f5dd93-85fd67b947 Error creating: pods "payload-processing-e2e-inf-a-f5dd93-85fd67b947-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f5dd93: serviceaccount "payload-processing-e2e-inf-a-f5dd93" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5 Created pod: payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq 44m Normal Pulled pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Started pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Container started 44m Normal Created pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Container created 44m Normal Created pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Add eth0 [10.132.0.129/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Container started 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Add eth0 [10.132.0.128/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Warning FailedCreate replicaset/payload-processing-e2e-api-a-022f62-6b6cb577c9 Error creating: pods "payload-processing-e2e-api-a-022f62-6b6cb577c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-022f62: serviceaccount "payload-processing-e2e-api-a-022f62" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-022f62-6b6cb577c9 Created pod: payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-022f62 Scaled up replica set payload-processing-e2e-api-a-022f62-6b6cb577c9 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f Error creating: pods "payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-022f62: serviceaccount "payload-processing-e2e-api-a-022f62" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f Created pod: payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-022f62 Scaled up replica set payload-pre-processing-e2e-api-a-022f62-6b68c9f45f from 0 to 1 44m Normal AddedInterface pod/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj Add eth0 [10.134.0.139/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp Add eth0 [10.134.0.138/23] from ovn-kubernetes 44m Normal Created pod/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp Container created 44m Normal Pulled pod/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj Container created 44m Normal Started pod/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj Container started 44m Normal Started pod/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp Container started 44m Normal Pulled pod/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk 44m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps 44m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 44m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-ghbc8 Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled down replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 9 to 7 44m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled down replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 10 to 7 44m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled down replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 9 to 7 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 7; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 7; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw8vk Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 7; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-jrvqn 44m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled down replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 9 to 7 44m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-dq55k 44m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-67wm4 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 7; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl 44m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled down replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 10 to 8 44m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 8; reason: All metrics below target 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 44m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 44m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-l8xws Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp 44m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-8c7sp Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-bjdzz Stopping container istio-proxy 44m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k 44m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Stopping container istio-proxy 44m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-8fqcl Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled down replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 9 to 7 44m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-xhbvc Stopping container istio-proxy 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 7; reason: All metrics below target 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 7; reason: All metrics below target 44m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-26tv8 Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled down replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 9 to 7 43m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-qpsps Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal SuccessfulCreate replicaset/e2e-api-a-166778-openshift-default-869f777c67 Created pod: e2e-api-a-166778-openshift-default-869f777c67-4ddgh 43m Normal ScalingReplicaSet deployment/e2e-api-a-166778-openshift-default Scaled up replica set e2e-api-a-166778-openshift-default-869f777c67 from 0 to 1 43m Normal AddedLabel gateway/e2e-api-a-166778 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-166778 43m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-lst2k Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal AddedInterface pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Add eth0 [10.133.0.165/23] from ovn-kubernetes 43m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-kcw79 Readiness probe error: Get "http://10.134.0.70:15021/healthz/ready": dial tcp 10.134.0.70:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-jqvc8 Readiness probe error: Get "http://10.134.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-166778 Scaled up replica set payload-pre-processing-e2e-api-a-166778-7d8fcb5659 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-166778-7d8fcb5659 Created pod: payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-166778-77d4bcfc6f Created pod: payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-166778-7d8fcb5659 Error creating: pods "payload-pre-processing-e2e-api-a-166778-7d8fcb5659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-166778: serviceaccount "payload-processing-e2e-api-a-166778" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-166778 Scaled up replica set payload-processing-e2e-api-a-166778-77d4bcfc6f from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-api-a-166778-77d4bcfc6f Error creating: pods "payload-processing-e2e-api-a-166778-77d4bcfc6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-166778: serviceaccount "payload-processing-e2e-api-a-166778" not found 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt Add eth0 [10.134.0.141/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Add eth0 [10.134.0.140/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt Container created 43m Normal Started pod/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt Container started 43m Normal Started pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Container started 43m Normal Created pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Container created 43m Normal Killing pod/payload-processing-e2e-api-a-5c6b75-58cdb8f699-tzrc2 Stopping container payload-processing 43m Normal Killing pod/payload-pre-processing-e2e-api-a-5c6b75-9f88c8787-vm79j Stopping container payload-pre-processing 43m Normal Started pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Container started 43m Normal Pulled pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Container created 43m Normal AddedLabel gateway/e2e-inf-b-375350 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-375350 43m Normal ScalingReplicaSet deployment/e2e-inf-b-375350-openshift-default Scaled up replica set e2e-inf-b-375350-openshift-default-5c64667db7 from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-inf-b-375350-openshift-default-5c64667db7 Created pod: e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 43m Normal AddedInterface pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Add eth0 [10.134.0.144/23] from ovn-kubernetes 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-80c4c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-80c4c8-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Startup probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 43m Warning ProbeError pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Startup probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-375350-68b888b95 Created pod: payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-375350-b98cbd657 Error creating: pods "payload-processing-e2e-inf-b-375350-b98cbd657-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-375350: serviceaccount "payload-processing-e2e-inf-b-375350" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-375350 Scaled up replica set payload-pre-processing-e2e-inf-b-375350-68b888b95 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-375350-b98cbd657 Created pod: payload-processing-e2e-inf-b-375350-b98cbd657-76rtq 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-375350-68b888b95 Error creating: pods "payload-pre-processing-e2e-inf-b-375350-68b888b95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-375350: serviceaccount "payload-processing-e2e-inf-b-375350" not found 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-5c6b75-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-375350 Scaled up replica set payload-processing-e2e-inf-b-375350-b98cbd657 from 0 to 1 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-5c6b75-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Started pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Container started 43m Normal Created pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Container created 43m Normal Created pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Container created 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Add eth0 [10.134.0.146/23] from ovn-kubernetes 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Add eth0 [10.134.0.147/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Container started 43m Normal Killing pod/payload-pre-processing-e2e-mt-76063d10-78f95fdf8c-lqtdt Stopping container payload-pre-processing 43m Normal Killing pod/payload-processing-e2e-mt-76063d10-f66f6f4bf-vrmjz Stopping container payload-processing 43m Normal ScalingReplicaSet deployment/e2e-api-b-58b0d1-openshift-default Scaled up replica set e2e-api-b-58b0d1-openshift-default-b5c977647 from 0 to 1 43m Normal AddedLabel gateway/e2e-api-b-58b0d1 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-58b0d1 43m Normal SuccessfulCreate replicaset/e2e-api-b-58b0d1-openshift-default-b5c977647 Created pod: e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n 43m Normal AddedInterface pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Add eth0 [10.134.0.148/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/e2e-inf-a-1c65e3-openshift-default-95896db5c Created pod: e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c Created pod: payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 43m Warning FailedCreate replicaset/payload-processing-e2e-api-b-58b0d1-786cf8cdb9 Error creating: pods "payload-processing-e2e-api-b-58b0d1-786cf8cdb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-58b0d1: serviceaccount "payload-processing-e2e-api-b-58b0d1" not found 43m Normal AddedLabel gateway/e2e-inf-a-1c65e3 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-1c65e3 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-58b0d1-786cf8cdb9 Created pod: payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv 43m Normal ScalingReplicaSet deployment/e2e-inf-a-1c65e3-openshift-default Scaled up replica set e2e-inf-a-1c65e3-openshift-default-95896db5c from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-58b0d1 Scaled up replica set payload-pre-processing-e2e-api-b-58b0d1-5455d7568c from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-58b0d1 Scaled up replica set payload-processing-e2e-api-b-58b0d1-786cf8cdb9 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c Error creating: pods "payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-58b0d1: serviceaccount "payload-processing-e2e-api-b-58b0d1" not found 43m Normal Pulled pod/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 Add eth0 [10.134.0.150/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv Container started 43m Normal Created pod/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 Container created 43m Normal AddedInterface pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Add eth0 [10.134.0.149/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 Container started 43m Normal Created pod/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv Container created 43m Normal Pulled pod/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv Add eth0 [10.134.0.151/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-1c65e3 Scaled up replica set payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7 from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-1c65e3 Scaled up replica set payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf Error creating: pods "payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1c65e3: serviceaccount "payload-processing-e2e-inf-a-1c65e3" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7 Error creating: pods "payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1c65e3: serviceaccount "payload-processing-e2e-inf-a-1c65e3" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf Created pod: payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7 Created pod: payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Add eth0 [10.134.0.153/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Container started 43m Normal Created pod/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc Container created 43m Normal Created pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Container created 43m Normal Started pod/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc Add eth0 [10.134.0.152/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Readiness probe error: HTTP probe failed with statuscode: 503... 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 7; reason: All metrics below target 43m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled down replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 8 to 7 43m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-wsrkx 43m Normal AddedLabel gateway/e2e-api-b-26310f Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-26310f 43m Normal ScalingReplicaSet deployment/e2e-api-b-26310f-openshift-default Scaled up replica set e2e-api-b-26310f-openshift-default-85bb76567f from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-api-b-26310f-openshift-default-85bb76567f Created pod: e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 43m Normal AddedInterface pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Add eth0 [10.134.0.155/23] from ovn-kubernetes 43m Normal Killing pod/e2e-api-a-5c6b75-openshift-default-86f46b878f-lrc9b Stopping container istio-proxy 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-80c4c8-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-80c4c8-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-26310f-58f7f46c57 Created pod: payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-26310f-778676cc8c Error creating: pods "payload-pre-processing-e2e-api-b-26310f-778676cc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-26310f: serviceaccount "payload-processing-e2e-api-b-26310f" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-26310f-778676cc8c Created pod: payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-26310f Scaled up replica set payload-pre-processing-e2e-api-b-26310f-778676cc8c from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-26310f Scaled up replica set payload-processing-e2e-api-b-26310f-58f7f46c57 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-api-b-26310f-58f7f46c57 Error creating: pods "payload-processing-e2e-api-b-26310f-58f7f46c57-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-26310f: serviceaccount "payload-processing-e2e-api-b-26310f" not found 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh Add eth0 [10.134.0.156/23] from ovn-kubernetes 43m Normal Created pod/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh Container created 43m Normal Started pod/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh Container started 43m Normal Pulled pod/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw Container started 43m Normal Created pod/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw Container created 43m Normal Pulled pod/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw Add eth0 [10.134.0.157/23] from ovn-kubernetes 42m Normal Killing pod/payload-processing-e2e-api-b-80c4c8-75974fbc49-zqjkh Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-api-b-80c4c8-5779b74587-k4gkg Stopping container payload-pre-processing 42m Normal Created pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Container created 42m Normal Started pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Container started 42m Normal Pulled pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Container created 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f 42m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Container started 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-76063d10-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Add eth0 [10.132.0.133/23] from ovn-kubernetes 42m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Container started 42m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Container created 42m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj 42m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Container created 42m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Add eth0 [10.133.0.170/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq 42m Normal ScalingReplicaSet deployment/e2e-mt-76063d10-openshift-default Scaled up replica set e2e-mt-76063d10-openshift-default-54bbfd48b9 from 1 to 4 42m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Add eth0 [10.132.0.132/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Container started 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-022f62-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning ProbeError pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Readiness probe error: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Pulled pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Container created 42m Normal Started pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Container started 42m Warning Unhealthy pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Readiness probe failed: Get "http://10.134.0.135:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-022f62-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal Pulled pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Container started 42m Normal Created pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Container created 42m Warning ProbeError pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Startup probe error: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Startup probe failed: Get "http://10.133.0.165:15021/healthz/ready": dial tcp 10.133.0.165:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Startup probe error: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Startup probe error: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Startup probe failed: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Startup probe failed: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Startup probe error: Get "http://10.132.0.132:15021/healthz/ready": dial tcp 10.132.0.132:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Startup probe failed: Get "http://10.132.0.133:15021/healthz/ready": dial tcp 10.132.0.133:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Startup probe failed: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused 42m Warning ProbeError pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Startup probe error: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Startup probe error: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Startup probe failed: Get "http://10.134.0.135:15021/healthz/ready": dial tcp 10.134.0.135:15021: connect: connection refused 42m Normal ScalingReplicaSet deployment/e2e-inf-b-2ce08a-openshift-default Scaled up replica set e2e-inf-b-2ce08a-openshift-default-9f569c5c5 from 0 to 1 42m Normal AddedLabel gateway/e2e-inf-b-2ce08a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-2ce08a 42m Normal SuccessfulCreate replicaset/e2e-inf-b-2ce08a-openshift-default-9f569c5c5 Created pod: e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 42m Normal AddedInterface pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Add eth0 [10.133.0.172/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd 42m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/e2e-mt-76063d10-openshift-default-54bbfd48b9 Created pod: e2e-mt-76063d10-openshift-default-54bbfd48b9-cvl2d 42m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Add eth0 [10.132.0.136/23] from ovn-kubernetes 42m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Add eth0 [10.133.0.173/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Add eth0 [10.132.0.135/23] from ovn-kubernetes 42m Normal Pulled pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-76063d10-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/e2e-mt-76063d10-openshift-default Scaled up replica set e2e-mt-76063d10-openshift-default-54bbfd48b9 from 4 to 8 42m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Container started 42m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Container started 42m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Container created 42m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Container created 42m Normal Created pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Container created 42m Normal Started pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Container started 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default New size: 4; reason: All metrics below target 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf Error creating: pods "payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2ce08a: serviceaccount "payload-processing-e2e-inf-b-2ce08a" not found 42m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Deleted pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db 42m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-nl9db Stopping container istio-proxy 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-2ce08a Scaled up replica set payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5 Created pod: payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-2ce08a Scaled up replica set payload-processing-e2e-inf-b-2ce08a-5bc49b65bf from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5 Error creating: pods "payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-2ce08a: serviceaccount "payload-processing-e2e-inf-b-2ce08a" not found 42m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled down replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 5 to 4 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf Created pod: payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr 42m Normal Pulled pod/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr Container started 42m Normal Created pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Container created 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Add eth0 [10.134.0.162/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Container started 42m Normal Killing pod/payload-processing-e2e-api-a-022f62-6b6cb577c9-w46bj Stopping container payload-processing 42m Normal Killing pod/payload-pre-processing-e2e-api-a-022f62-6b68c9f45f-ls8hp Stopping container payload-pre-processing 42m Normal Created pod/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr Container created 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr Add eth0 [10.134.0.161/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Startup probe error: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Startup probe failed: Get "http://10.132.0.136:15021/healthz/ready": dial tcp 10.132.0.136:15021: connect: connection refused 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Startup probe error: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Startup probe failed: Get "http://10.132.0.135:15021/healthz/ready": dial tcp 10.132.0.135:15021: connect: connection refused 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Stopping container istio-proxy 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-p8m6f Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Stopping container istio-proxy 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Readiness probe error: HTTP probe failed with statuscode: 503... 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-lvkgq Stopping container istio-proxy 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Stopping container istio-proxy 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-8wkjd Stopping container istio-proxy 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-pzhrj Readiness probe failed: HTTP probe failed with statuscode: 503 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-4x8c8 Stopping container istio-proxy 42m Normal Killing pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-rtfk6 Stopping container istio-proxy 42m Warning ProbeError pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-mt-76063d10-openshift-default-54bbfd48b9-kq55m Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f5dd93-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f5dd93-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Normal ScalingReplicaSet deployment/e2e-inf-a-784444-openshift-default Scaled up replica set e2e-inf-a-784444-openshift-default-c75d4cbd5 from 0 to 1 42m Normal AddedLabel gateway/e2e-inf-a-784444 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-784444 42m Normal Killing pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Stopping container payload-processing 42m Normal SuccessfulCreate replicaset/e2e-inf-a-784444-openshift-default-c75d4cbd5 Created pod: e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp 42m Normal Killing pod/payload-pre-processing-e2e-api-a-166778-7d8fcb5659-jq7tt Stopping container payload-pre-processing 42m Normal Killing pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Stopping container istio-proxy 42m Normal AddedInterface pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Add eth0 [10.132.0.137/23] from ovn-kubernetes 42m Normal AddedLabel gateway/e2e-inf-a-2001b0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-2001b0 42m Normal ScalingReplicaSet deployment/e2e-inf-a-2001b0-openshift-default Scaled up replica set e2e-inf-a-2001b0-openshift-default-ffb8fbb5c from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c Created pod: e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q 42m Normal AddedInterface pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Add eth0 [10.133.0.176/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-784444 Scaled up replica set payload-pre-processing-e2e-inf-a-784444-79c887957f from 0 to 1 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-784444 Scaled up replica set payload-processing-e2e-inf-a-784444-55c46475 from 0 to 1 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f5dd93-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f5dd93-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-784444-79c887957f Error creating: pods "payload-pre-processing-e2e-inf-a-784444-79c887957f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-784444: serviceaccount "payload-processing-e2e-inf-a-784444" not found 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-784444-55c46475 Error creating: pods "payload-processing-e2e-inf-a-784444-55c46475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-784444: serviceaccount "payload-processing-e2e-inf-a-784444" not found 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-2001b0 Scaled up replica set payload-pre-processing-e2e-inf-a-2001b0-86fcc565f from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-784444-79c887957f Created pod: payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-2001b0 Scaled up replica set payload-processing-e2e-inf-a-2001b0-7f4d46cd5f from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-784444-55c46475 Created pod: payload-processing-e2e-inf-a-784444-55c46475-d78lt 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f Error creating: pods "payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2001b0: serviceaccount "payload-processing-e2e-inf-a-2001b0" not found 42m Normal Created pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Add eth0 [10.133.0.178/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Container created 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Add eth0 [10.133.0.177/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f Created pod: payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g 42m Warning ProbeError pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Readiness probe error: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused... 42m Normal Started pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Container started 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f Created pod: payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 42m Normal Pulled pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f Error creating: pods "payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-2001b0: serviceaccount "payload-processing-e2e-inf-a-2001b0" not found 42m Normal Started pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Container started 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/e2e-api-b-80c4c8-openshift-default-78ff9c847-vj9ph Readiness probe failed: Get "http://10.134.0.130:15021/healthz/ready": dial tcp 10.134.0.130:15021: connect: connection refused 42m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Readiness probe failed: timeout: failed to connect service "10.134.0.106:50052" within 1s: context deadline exceeded 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-022f62-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-022f62-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Add eth0 [10.134.0.163/23] from ovn-kubernetes 41m Warning Unhealthy pod/payload-processing-e2e-api-a-166778-77d4bcfc6f-88bhr Readiness probe failed: timeout: failed to connect service "10.134.0.140:50052" within 1s: context deadline exceeded 41m Normal Pulled pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Container started 41m Normal AddedLabel gateway/e2e-inf-a-04f248 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-04f248 41m Normal Created pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Container created 41m Normal Started pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Container started 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Add eth0 [10.134.0.164/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Container created 41m Normal SuccessfulCreate replicaset/e2e-inf-a-04f248-openshift-default-7d6557f5c5 Created pod: e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-166778-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-166778-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal ScalingReplicaSet deployment/e2e-inf-a-04f248-openshift-default Scaled up replica set e2e-inf-a-04f248-openshift-default-7d6557f5c5 from 0 to 1 41m Normal AddedInterface pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Add eth0 [10.134.0.165/23] from ovn-kubernetes 41m Warning Unhealthy pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Readiness probe failed: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Readiness probe error: Get "http://10.134.0.148:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Normal SuccessfulCreate replicaset/e2e-inf-b-375350-openshift-default-5c64667db7 Created pod: e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk 41m Normal SuccessfulCreate replicaset/e2e-inf-b-375350-openshift-default-5c64667db7 Created pod: e2e-inf-b-375350-openshift-default-5c64667db7-664vq 41m Normal ScalingReplicaSet deployment/e2e-inf-b-375350-openshift-default Scaled up replica set e2e-inf-b-375350-openshift-default-5c64667db7 from 1 to 3 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-375350-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 41m Normal Started pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Container started 41m Normal AddedInterface pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Add eth0 [10.132.0.139/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Add eth0 [10.133.0.179/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Container created 41m Normal Started pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Container started 41m Normal Created pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Container created 41m Normal Pulled pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf Created pod: payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-04f248 Scaled up replica set payload-pre-processing-e2e-inf-a-04f248-65b9669fcf from 0 to 1 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-04f248-6f57f7dc7 Error creating: pods "payload-processing-e2e-inf-a-04f248-6f57f7dc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-04f248: serviceaccount "payload-processing-e2e-inf-a-04f248" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-04f248-6f57f7dc7 Created pod: payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf Error creating: pods "payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-04f248: serviceaccount "payload-processing-e2e-inf-a-04f248" not found 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-04f248 Scaled up replica set payload-processing-e2e-inf-a-04f248-6f57f7dc7 from 0 to 1 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Add eth0 [10.134.0.166/23] from ovn-kubernetes 41m Warning ProbeError pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Readiness probe error: Get "http://10.134.0.144:15021/healthz/ready": read tcp 10.134.0.2:47700->10.134.0.144:15021: read: connection reset by peer... 41m Warning Unhealthy pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Readiness probe failed: Get "http://10.134.0.144:15021/healthz/ready": read tcp 10.134.0.2:47700->10.134.0.144:15021: read: connection reset by peer 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Add eth0 [10.134.0.167/23] from ovn-kubernetes 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Pulled pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Container created 41m Normal Started pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Container started 41m Normal Created pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Container created 41m Normal Started pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Container started 41m Normal Pulled pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Container started 41m Normal Created pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Container created 41m Normal Pulled pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Container created 41m Normal Started pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Container started 41m Normal Created pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Container created 41m Normal Started pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Container started 41m Warning ProbeError pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Startup probe error: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Startup probe failed: Get "http://10.132.0.139:15021/healthz/ready": dial tcp 10.132.0.139:15021: connect: connection refused 41m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Readiness probe failed: timeout: failed to connect service "10.134.0.114:50052" within 1s: context deadline exceeded 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-784444-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-784444-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-375350-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-375350-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning Unhealthy pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Startup probe failed: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused 41m Warning ProbeError pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1c65e3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning ProbeError pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Startup probe error: Get "http://10.134.0.149:15021/healthz/ready": dial tcp 10.134.0.149:15021: connect: connection refused... 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1c65e3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Killing pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Stopping container istio-proxy 41m Warning Unhealthy pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-api-a-022f62-openshift-default-c7698fd8f-24f5p Readiness probe failed: HTTP probe failed with statuscode: 503 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f5dd93-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-inf-a-f5dd93-openshift-default Scaled up replica set e2e-inf-a-f5dd93-openshift-default-86c6c56f5c from 1 to 2 41m Normal SuccessfulCreate replicaset/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c Created pod: e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s 41m Normal Created pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Container created 41m Normal Started pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Container started 41m Normal AddedInterface pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Add eth0 [10.133.0.181/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default Scaled down replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 7 to 6 41m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kh5rv Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default New size: 3; reason: All metrics below target 41m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m Stopping container istio-proxy 41m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Deleted pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-ffx4m 41m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled down replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 4 to 3 41m Normal Killing pod/payload-processing-e2e-api-b-58b0d1-786cf8cdb9-cbftv Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-api-b-58b0d1-5455d7568c-jh7m4 Stopping container payload-pre-processing 41m Warning ProbeError pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-375350-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-375350-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal Killing pod/e2e-api-a-166778-openshift-default-869f777c67-4ddgh Stopping container istio-proxy 41m Normal AddedLabel gateway/e2e-inf-b-99afca Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-99afca 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-58b0d1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/e2e-inf-b-99afca-openshift-default-84d498d4ff Created pod: e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn 41m Normal ScalingReplicaSet deployment/e2e-inf-b-99afca-openshift-default Scaled up replica set e2e-inf-b-99afca-openshift-default-84d498d4ff from 0 to 1 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-58b0d1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal ScalingReplicaSet deployment/e2e-inf-b-99e837-openshift-default Scaled up replica set e2e-inf-b-99e837-openshift-default-7bc77d59c5 from 0 to 1 41m Normal AddedLabel gateway/e2e-inf-b-99e837 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-99e837 41m Normal SuccessfulCreate replicaset/e2e-inf-b-99e837-openshift-default-7bc77d59c5 Created pod: e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm 41m Normal AddedInterface pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Add eth0 [10.134.0.172/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Add eth0 [10.134.0.173/23] from ovn-kubernetes 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-99afca Scaled up replica set payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75 from 0 to 1 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1c65e3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-99afca Scaled up replica set payload-processing-e2e-inf-b-99afca-5bc4ccf5cc from 0 to 1 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1c65e3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc Error creating: pods "payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-99afca: serviceaccount "payload-processing-e2e-inf-b-99afca" not found 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75 Error creating: pods "payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-99afca: serviceaccount "payload-processing-e2e-inf-b-99afca" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75 Created pod: payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc Created pod: payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg 41m Normal Created pod/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg Container created 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-99e837 Scaled up replica set payload-processing-e2e-inf-b-99e837-75d8d9f76f from 0 to 1 41m Normal AddedInterface pod/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg Add eth0 [10.134.0.175/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Add eth0 [10.134.0.174/23] from ovn-kubernetes 41m Normal Started pod/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg Container started 41m Normal Pulled pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Container created 41m Normal Started pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Container started 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87 Error creating: pods "payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-99e837: serviceaccount "payload-processing-e2e-inf-b-99e837" not found 41m Normal Killing pod/payload-processing-e2e-api-b-26310f-58f7f46c57-hqjhw Stopping container payload-processing 41m Normal Killing pod/payload-pre-processing-e2e-api-b-26310f-778676cc8c-ppxzh Stopping container payload-pre-processing 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-99e837 Scaled up replica set payload-pre-processing-e2e-inf-b-99e837-84bbb74c87 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87 Created pod: payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-99e837-75d8d9f76f Error creating: pods "payload-processing-e2e-inf-b-99e837-75d8d9f76f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-99e837: serviceaccount "payload-processing-e2e-inf-b-99e837" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-99e837-75d8d9f76f Created pod: payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 41m Normal AddedInterface pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Add eth0 [10.133.0.184/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Container created 41m Normal Pulled pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Add eth0 [10.133.0.183/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Container started 41m Normal Started pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Container started 41m Normal Created pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Container created 41m Normal Pulled pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal SuccessfulCreate replicaset/e2e-inf-b-10b00a-openshift-default-55c579877c Created pod: e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z 41m Normal ScalingReplicaSet deployment/e2e-inf-b-10b00a-openshift-default Scaled up replica set e2e-inf-b-10b00a-openshift-default-55c579877c from 0 to 1 41m Normal AddedLabel gateway/e2e-inf-b-10b00a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-10b00a 41m Normal AddedInterface pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Add eth0 [10.133.0.185/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Readiness probe failed: Get "http://10.133.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Readiness probe error: Get "http://10.133.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-10b00a-659db85879 Created pod: payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f Error creating: pods "payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-10b00a: serviceaccount "payload-processing-e2e-inf-b-10b00a" not found 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-10b00a-659db85879 Error creating: pods "payload-processing-e2e-inf-b-10b00a-659db85879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-10b00a: serviceaccount "payload-processing-e2e-inf-b-10b00a" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-10b00a Scaled up replica set payload-processing-e2e-inf-b-10b00a-659db85879 from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-10b00a Scaled up replica set payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f Created pod: payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p 40m Normal Pulled pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Container started 40m Normal Started pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Container started 40m Normal Created pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Container created 40m Normal Pulled pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Add eth0 [10.134.0.176/23] from ovn-kubernetes 40m Normal Created pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Container created 40m Normal AddedInterface pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Add eth0 [10.134.0.177/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Container created 40m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Deleted pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc 40m Normal Created pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Container created 40m Normal Started pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Container started 40m Normal Pulled pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Container created 40m Warning Unhealthy pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Readiness probe failed: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Normal Started pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Container started 40m Warning ProbeError pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Readiness probe error: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n Stopping container istio-proxy 40m Normal Pulled pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-kmsmc Stopping container istio-proxy 40m Normal Pulled pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled down replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 5 to 3 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default New size: 3; reason: All metrics below target 40m Normal Started pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Container started 40m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Deleted pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-jf99n 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-2001b0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-2001b0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Pulled pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Container started 40m Normal Created pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Container created 40m Warning Unhealthy pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Startup probe failed: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Startup probe error: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Startup probe failed: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Startup probe error: Get "http://10.133.0.172:15021/healthz/ready": dial tcp 10.133.0.172:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused 40m Warning ProbeError pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-99afca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-99afca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Killing pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Stopping container istio-proxy 40m Normal SuccessfulCreate replicaset/e2e-inf-a-a4240a-openshift-default-8449487f59 Created pod: e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc 40m Normal ScalingReplicaSet deployment/e2e-inf-a-a4240a-openshift-default Scaled up replica set e2e-inf-a-a4240a-openshift-default-8449487f59 from 0 to 1 40m Warning ProbeError pod/e2e-api-b-58b0d1-openshift-default-b5c977647-ltl2n Readiness probe error: HTTP probe failed with statuscode: 503... 40m Normal AddedLabel gateway/e2e-inf-a-a4240a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-a4240a 40m Normal AddedInterface pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Add eth0 [10.134.0.178/23] from ovn-kubernetes 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-a4240a-745976f465 Created pod: payload-processing-e2e-inf-a-a4240a-745976f465-st69v 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf Error creating: pods "payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a4240a: serviceaccount "payload-processing-e2e-inf-a-a4240a" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-a4240a Scaled up replica set payload-processing-e2e-inf-a-a4240a-745976f465 from 0 to 1 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2ce08a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-a4240a-745976f465 Error creating: pods "payload-processing-e2e-inf-a-a4240a-745976f465-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-a4240a: serviceaccount "payload-processing-e2e-inf-a-a4240a" not found 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-a4240a Scaled up replica set payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf from 0 to 1 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2ce08a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf Created pod: payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Add eth0 [10.134.0.180/23] from ovn-kubernetes 40m Normal AddedInterface pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Add eth0 [10.134.0.179/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Created pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Container created 40m Normal Created pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Container created 40m Normal Started pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Container started 40m Normal Started pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Container started 40m Normal Pulled pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-26310f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-26310f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-784444-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-784444-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-2001b0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-2001b0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-2ce08a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-2ce08a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal Killing pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Stopping container istio-proxy 40m Normal SuccessfulCreate replicaset/e2e-inf-a-9e044c-openshift-default-85757fb4b9 Created pod: e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl 40m Normal AddedLabel gateway/e2e-inf-a-9e044c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-9e044c 40m Normal ScalingReplicaSet deployment/e2e-inf-a-9e044c-openshift-default Scaled up replica set e2e-inf-a-9e044c-openshift-default-85757fb4b9 from 0 to 1 40m Normal AddedInterface pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Add eth0 [10.134.0.182/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Readiness probe error: HTTP probe failed with statuscode: 503... 40m Warning Unhealthy pod/e2e-api-b-26310f-openshift-default-85bb76567f-kw9m5 Readiness probe failed: HTTP probe failed with statuscode: 503 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-9e044c-668c79d887 Created pod: payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-9e044c-75ffc66cdc Error creating: pods "payload-processing-e2e-inf-a-9e044c-75ffc66cdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9e044c: serviceaccount "payload-processing-e2e-inf-a-9e044c" not found 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-9e044c-668c79d887 Error creating: pods "payload-pre-processing-e2e-inf-a-9e044c-668c79d887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-9e044c: serviceaccount "payload-processing-e2e-inf-a-9e044c" not found 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-9e044c Scaled up replica set payload-processing-e2e-inf-a-9e044c-75ffc66cdc from 0 to 1 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-9e044c-75ffc66cdc Created pod: payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-9e044c Scaled up replica set payload-pre-processing-e2e-inf-a-9e044c-668c79d887 from 0 to 1 40m Normal AddedInterface pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Add eth0 [10.133.0.188/23] from ovn-kubernetes 40m Normal Pulled pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Container started 40m Normal Created pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Container created 40m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Readiness probe failed: timeout: health rpc did not complete within 1s 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Add eth0 [10.134.0.184/23] from ovn-kubernetes 40m Warning Unhealthy pod/payload-processing-55fbbbf9dc-l5qnm Readiness probe failed: timeout: health rpc did not complete within 1s 40m Normal Pulled pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal Started pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Container started 40m Normal Created pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Container created 39m Normal SuccessfulCreate replicaset/e2e-inf-b-99afca-openshift-default-84d498d4ff Created pod: e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv 39m Normal ScalingReplicaSet deployment/e2e-inf-b-99afca-openshift-default Scaled up replica set e2e-inf-b-99afca-openshift-default-84d498d4ff from 1 to 4 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-99afca-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 39m Normal SuccessfulCreate replicaset/e2e-inf-b-99afca-openshift-default-84d498d4ff Created pod: e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg 39m Normal SuccessfulCreate replicaset/e2e-inf-b-99afca-openshift-default-84d498d4ff Created pod: e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 39m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Readiness probe error: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Readiness probe failed: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Readiness probe error: Get "http://10.134.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Readiness probe failed: Get "http://10.134.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Warning ProbeError pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Readiness probe error: Get "http://10.134.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Readiness probe failed: Get "http://10.134.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal Pulled pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Created pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Container created 39m Normal Pulled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Container started 39m Normal Created pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Container created 39m Normal Started pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Container started 39m Normal Created pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Container created 39m Normal Started pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Container started 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9e044c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9e044c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-99e837-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-99e837-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-04f248-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-04f248-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning Unhealthy pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Startup probe failed: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused 39m Normal AddedLabel gateway/e2e-inf-b-0c091d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-0c091d 39m Warning ProbeError pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Startup probe error: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/e2e-inf-b-0c091d-openshift-default-6855f44f99 Created pod: e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm 39m Normal ScalingReplicaSet deployment/e2e-inf-b-0c091d-openshift-default Scaled up replica set e2e-inf-b-0c091d-openshift-default-6855f44f99 from 0 to 1 39m Warning Unhealthy pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Startup probe failed: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Startup probe error: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-99afca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-99afca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-99e837-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-99e837-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-04f248-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-04f248-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 6; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled down replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 7 to 6 39m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled down replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 7 to 6 39m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Readiness probe failed: HTTP probe failed with statuscode: 503 39m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf 39m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r96sf Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 5; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default Scaled down replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 7 to 6 39m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 6; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-5zvm7 39m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-tjhs7 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 6; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled down replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 7 to 6 39m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled down replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 7 to 6 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 6; reason: All metrics below target 39m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-lgmlv Stopping container istio-proxy 39m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-jjk7f Stopping container istio-proxy 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 6; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr 39m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-wwkmr Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-qdrlt 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 6; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default Scaled down replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 7 to 6 38m Warning ProbeError pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Readiness probe error: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Readiness probe failed: Get "http://10.134.0.182:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning Unhealthy pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Readiness probe failed: Get "http://10.133.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Readiness probe error: Get "http://10.133.0.185:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Normal Created pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Container created 38m Normal Started pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Container started 38m Normal Pulled pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Add eth0 [10.133.0.192/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Started pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Container started 38m Normal Created pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Container created 38m Normal Pulled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Container created 38m Normal AddedInterface pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Add eth0 [10.132.0.147/23] from ovn-kubernetes 38m Normal AddedInterface pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Add eth0 [10.133.0.191/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Add eth0 [10.134.0.188/23] from ovn-kubernetes 38m Normal Started pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Container started 38m Normal Pulled pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Container created 38m Normal Created pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Container created 38m Normal Started pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Container started 38m Normal Started pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Container started 38m Normal Pulled pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Container created 38m Normal Started pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Container started 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-0c091d-7d79b764b9 Error creating: pods "payload-processing-e2e-inf-b-0c091d-7d79b764b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0c091d: serviceaccount "payload-processing-e2e-inf-b-0c091d" not found 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c Error creating: pods "payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0c091d: serviceaccount "payload-processing-e2e-inf-b-0c091d" not found 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c Created pod: payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-0c091d Scaled up replica set payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c from 0 to 1 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-0c091d Scaled up replica set payload-processing-e2e-inf-b-0c091d-7d79b764b9 from 0 to 1 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-0c091d-7d79b764b9 Created pod: payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg 38m Warning Unhealthy pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Startup probe failed: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Startup probe error: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Startup probe failed: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Startup probe error: Get "http://10.132.0.147:15021/healthz/ready": dial tcp 10.132.0.147:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Startup probe failed: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Startup probe error: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused... 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-10b00a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-10b00a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning ProbeError pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Startup probe error: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Startup probe failed: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Startup probe error: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Startup probe failed: Get "http://10.134.0.188:15021/healthz/ready": dial tcp 10.134.0.188:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/e2e-inf-b-99e837-openshift-default Scaled up replica set e2e-inf-b-99e837-openshift-default-7bc77d59c5 from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-99e837-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-inf-b-99e837-openshift-default-7bc77d59c5 Created pod: e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 38m Warning Unhealthy pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Startup probe failed: Get "http://10.134.0.182:15021/healthz/ready": dial tcp 10.134.0.182:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Startup probe error: Get "http://10.134.0.182:15021/healthz/ready": dial tcp 10.134.0.182:15021: connect: connection refused... 38m Normal SuccessfulCreate replicaset/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c Created pod: e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 38m Normal ScalingReplicaSet deployment/e2e-inf-a-2001b0-openshift-default Scaled up replica set e2e-inf-a-2001b0-openshift-default-ffb8fbb5c from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-2001b0-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a4240a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a4240a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Add eth0 [10.132.0.161/23] from ovn-kubernetes 38m Normal ScalingReplicaSet deployment/e2e-inf-b-ae79df-openshift-default Scaled up replica set e2e-inf-b-ae79df-openshift-default-777549df6f from 0 to 1 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Container created 38m Normal Started pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Container started 38m Normal AddedLabel gateway/e2e-inf-b-ae79df Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-ae79df 38m Normal SuccessfulCreate replicaset/e2e-inf-b-ae79df-openshift-default-777549df6f Created pod: e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-10b00a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-10b00a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg Add eth0 [10.132.0.162/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg Container created 38m Normal Pulled pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Started pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg Container started 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-9e044c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-9e044c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-a4240a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-a4240a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-q757x Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-q757x 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-r5m84 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 10 to 8 37m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-r5m84 Stopping container istio-proxy 37m Normal AddedInterface pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Add eth0 [10.134.0.200/23] from ovn-kubernetes 37m Normal Pulled pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Add eth0 [10.134.0.201/23] from ovn-kubernetes 37m Normal Started pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Container started 37m Normal Created pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Container created 37m Normal Pulled pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Add eth0 [10.132.0.165/23] from ovn-kubernetes 37m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-ae79df-54c86dbff Error creating: pods "payload-processing-e2e-inf-b-ae79df-54c86dbff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ae79df: serviceaccount "payload-processing-e2e-inf-b-ae79df" not found 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-ae79df Scaled up replica set payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59 from 0 to 1 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-ae79df-54c86dbff Created pod: payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59 Created pod: payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59 Error creating: pods "payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-ae79df: serviceaccount "payload-processing-e2e-inf-b-ae79df" not found 37m Normal Started pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Container started 37m Normal Created pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Container created 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-ae79df Scaled up replica set payload-processing-e2e-inf-b-ae79df-54c86dbff from 0 to 1 37m Warning Unhealthy pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Startup probe failed: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Startup probe error: Get "http://10.134.0.201:15021/healthz/ready": dial tcp 10.134.0.201:15021: connect: connection refused... 37m Normal Pulled pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal Created pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Container created 37m Normal Started pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Container started 37m Normal SuccessfulCreate replicaset/e2e-inf-a-a4240a-openshift-default-8449487f59 Created pod: e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-a4240a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal ScalingReplicaSet deployment/e2e-inf-a-a4240a-openshift-default Scaled up replica set e2e-inf-a-a4240a-openshift-default-8449487f59 from 1 to 2 37m Warning ProbeError pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Startup probe error: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused... 37m Warning Unhealthy pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Startup probe failed: Get "http://10.133.0.192:15021/healthz/ready": dial tcp 10.133.0.192:15021: connect: connection refused 36m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-fv7j6 36m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-fv7j6 Stopping container discovery 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 5 to 4 36m Normal Started pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Container started 36m Normal AddedInterface pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Add eth0 [10.133.0.197/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Add eth0 [10.133.0.194/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Container created 36m Normal Started pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 Container started 36m Normal Created pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 Container created 36m Normal AddedInterface pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 Add eth0 [10.133.0.196/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Container created 36m Normal Started pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Container started 36m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0c091d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0c091d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Startup probe failed: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Startup probe error: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Startup probe failed: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Startup probe error: Get "http://10.133.0.194:15021/healthz/ready": dial tcp 10.133.0.194:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Readiness probe error: Get "http://10.134.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Readiness probe failed: Get "http://10.134.0.200:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z 36m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-fp42z Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default New size: 5; reason: All metrics below target 35m Normal Pulled pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Created pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Container created 35m Normal Started pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Container started 35m Warning ProbeError pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Startup probe error: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused... 35m Warning Unhealthy pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Startup probe failed: Get "http://10.134.0.200:15021/healthz/ready": dial tcp 10.134.0.200:15021: connect: connection refused 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ae79df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ae79df-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s 35m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default Scaled down replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 6 to 4 35m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default Scaled down replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 6 to 5 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 5; reason: All metrics below target 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 4; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default New size: 5; reason: All metrics below target 35m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r 35m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m 35m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-5xx6s Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-7vzcq 35m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-64w4r Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-g8x6m Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default New size: 4; reason: All metrics below target 35m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-76lsq Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Deleted pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-76nz6 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default New size: 2; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled down replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 3 to 2 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Stopping container payload-processing 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-ae79df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-ae79df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 35m Normal Killing pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-jrh88 Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Stopping container istio-proxy 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-f5dd93-85fd67b947-4wgxc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-f5dd93-5c967b5ff5-8gksq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning ProbeError pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-a-f5dd93-openshift-default-86c6c56f5c-6pc6s Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/payload-processing-e2e-inf-a-1c65e3-54b4ddd7d7-lbrcc Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Stopping container payload-pre-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Stopping container payload-processing 35m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv 35m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 35m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-rbqs4 Stopping container istio-proxy 35m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-zn5vv Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 4; reason: All metrics below target 35m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default Scaled down replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 6 to 4 35m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5x7vq 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 34m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd Stopping container istio-proxy 34m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-7s2p8 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal SuccessfulDelete replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Deleted pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-rj7xd 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default New size: 2; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled down replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 3 to 2 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-klsmb 34m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-klsmb Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 8 to 7 34m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 34m Normal Killing pod/e2e-inf-a-1c65e3-openshift-default-95896db5c-5kw6m Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw Stopping container istio-proxy 34m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-k66xw 34m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-9gnt8 34m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr 34m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-k7wzc 34m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default Scaled down replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 6 to 4 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 4; reason: All metrics below target 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-1c65e3-77686b98cf-wzm9h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/e2e-inf-b-375350-openshift-default-5c64667db7-664vq Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-375350-openshift-default-5c64667db7-b7gw6 Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-375350-openshift-default-5c64667db7-dr7lk Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 7 to 6 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-whjkj 34m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-whjkj Stopping container istio-proxy 34m Warning Unhealthy pod/payload-processing-e2e-inf-b-375350-b98cbd657-76rtq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-375350-68b888b95-4mx6d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-n6vmr Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-b-2ce08a-5bc49b65bf-qn2rr Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Stopping container payload-processing 34m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Stopping container istio-proxy 34m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Stopping container payload-processing 34m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-ddbpq Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Killing pod/e2e-inf-a-784444-openshift-default-c75d4cbd5-dd5kp Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-2ce08a-openshift-default-9f569c5c5-6xrv9 Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-jb68q Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Stopping container istio-proxy 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc Created pod: e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-7225e5-openshift-default Scaled up replica set e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc from 0 to 1 34m Normal AddedLabel gateway/e2e-shared-a-w2-7225e5 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-7225e5 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-2ce08a-69744fddc5-cfpqd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal AddedInterface pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Add eth0 [10.134.0.205/23] from ovn-kubernetes 34m Warning ProbeError pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/e2e-inf-a-2001b0-openshift-default-ffb8fbb5c-hs246 Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65 Error creating: pods "payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-7225e5: serviceaccount "payload-processing-e2e-shared-a-w2-7225e5" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c Created pod: payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-7225e5 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c from 0 to 1 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65 Created pod: payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c Error creating: pods "payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-7225e5: serviceaccount "payload-processing-e2e-shared-a-w2-7225e5" not found 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-7225e5 Scaled up replica set payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65 from 0 to 1 34m Normal Pulled pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Container created 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Add eth0 [10.134.0.206/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Add eth0 [10.134.0.207/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Container started 34m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Container started 34m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Container created 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-784444-79c887957f-gsjc5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/payload-processing-e2e-inf-b-99afca-5bc4ccf5cc-qs8gg Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Stopping container payload-pre-processing 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-784444-55c46475-d78lt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Stopping container payload-processing 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-2001b0-86fcc565f-jqsc2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-2001b0-7f4d46cd5f-zlt2g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal Killing pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-kr4hn Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-mx6kv Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-grqzm Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-99e837-openshift-default-7bc77d59c5-hlfr9 Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-dw8pg Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-b-99afca-openshift-default-84d498d4ff-8ng62 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Normal Started pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Container started 33m Normal ScalingReplicaSet deployment/e2e-inf-b-10b00a-openshift-default Scaled up replica set e2e-inf-b-10b00a-openshift-default-55c579877c from 1 to 2 33m Normal Pulled pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Add eth0 [10.132.0.167/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-inf-a-9e044c-openshift-default-85757fb4b9 Created pod: e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv 33m Normal Created pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Container created 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-99afca-6df5cd4d75-kqpvz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-10b00a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-inf-a-9e044c-openshift-default Scaled up replica set e2e-inf-a-9e044c-openshift-default-85757fb4b9 from 1 to 2 33m Normal SuccessfulCreate replicaset/e2e-inf-b-10b00a-openshift-default-55c579877c Created pod: e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-9e044c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal AddedInterface pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Add eth0 [10.132.0.168/23] from ovn-kubernetes 33m Normal Started pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Container started 33m Normal Created pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Container created 33m Normal Pulled pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 33m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 6 to 5 33m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-kkcz5 Stopping container istio-proxy 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-99e837-84bbb74c87-vt2jq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-kkcz5 33m Warning Unhealthy pod/payload-processing-e2e-inf-b-99e837-75d8d9f76f-5r7g9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Startup probe failed: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused 33m Warning ProbeError pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Startup probe error: Get "http://10.132.0.167:15021/healthz/ready": dial tcp 10.132.0.167:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Startup probe error: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Startup probe failed: Get "http://10.132.0.168:15021/healthz/ready": dial tcp 10.132.0.168:15021: connect: connection refused 33m Normal AddedLabel gateway/e2e-shared-b-w2-c78897 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-c78897 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-c78897-openshift-default Scaled up replica set e2e-shared-b-w2-c78897-openshift-default-7c9787498f from 0 to 1 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b 33m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Add eth0 [10.134.0.211/23] from ovn-kubernetes 33m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-c78897-68dd595655 Error creating: pods "payload-processing-e2e-shared-b-w2-c78897-68dd595655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-c78897: serviceaccount "payload-processing-e2e-shared-b-w2-c78897" not found 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-c78897 Scaled up replica set payload-processing-e2e-shared-b-w2-c78897-68dd595655 from 0 to 1 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-c78897 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c from 0 to 1 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-c78897-68dd595655 Created pod: payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c Error creating: pods "payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-c78897: serviceaccount "payload-processing-e2e-shared-b-w2-c78897" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c Created pod: payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb 33m Normal Started pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Container started 33m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Container created 33m Normal Created pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Container created 33m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Add eth0 [10.133.0.217/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Container started 33m Normal Pulled pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Add eth0 [10.133.0.216/23] from ovn-kubernetes 33m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal Killing pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Stopping container payload-pre-processing 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-c8d121-openshift-default Scaled up replica set e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c from 1 to 2 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w0-c8d121-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c Created pod: e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl 32m Normal Pulled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Container created 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 Created pod: e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-383fde-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-383fde-openshift-default Scaled up replica set e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 from 1 to 2 32m Normal AddedInterface pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Add eth0 [10.134.0.214/23] from ovn-kubernetes 32m Normal Created pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Container created 32m Normal Started pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Container started 32m Normal AddedInterface pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Add eth0 [10.134.0.213/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Container started 32m Warning Unhealthy pod/payload-processing-e2e-inf-a-04f248-6f57f7dc7-snmtt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-04f248-65b9669fcf-7fzhs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Killing pod/e2e-inf-a-04f248-openshift-default-7d6557f5c5-vm6dj Stopping container istio-proxy 32m Warning Unhealthy pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Startup probe failed: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Startup probe error: Get "http://10.134.0.213:15021/healthz/ready": dial tcp 10.134.0.213:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Startup probe error: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Startup probe failed: Get "http://10.134.0.214:15021/healthz/ready": dial tcp 10.134.0.214:15021: connect: connection refused 32m Normal Killing pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Stopping container payload-processing 32m Normal Killing pod/e2e-inf-b-10b00a-openshift-default-55c579877c-bvsrc Stopping container istio-proxy 32m Normal Killing pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Stopping container istio-proxy 32m Normal AddedLabel gateway/e2e-shared-a-w1-5ccd7b Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-5ccd7b 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-5ccd7b-openshift-default Scaled up replica set e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f from 0 to 1 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f Created pod: e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 32m Normal AddedInterface pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Add eth0 [10.134.0.217/23] from ovn-kubernetes 32m Warning ProbeError pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Readiness probe error: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-inf-b-10b00a-openshift-default-55c579877c-krd5z Readiness probe failed: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c Error creating: pods "payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-5ccd7b: serviceaccount "payload-processing-e2e-shared-a-w1-5ccd7b" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-5ccd7b Scaled up replica set payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999 Created pod: payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-5ccd7b: serviceaccount "payload-processing-e2e-shared-a-w1-5ccd7b" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c Created pod: payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-5ccd7b Scaled up replica set payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999 from 0 to 1 32m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Add eth0 [10.133.0.224/23] from ovn-kubernetes 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Add eth0 [10.133.0.225/23] from ovn-kubernetes 32m Normal Pulled pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Container created 32m Normal Created pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Container created 32m Normal Started pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Container started 32m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Container started 32m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-c78897-openshift-default Scaled up replica set e2e-shared-b-w2-c78897-openshift-default-7c9787498f from 1 to 4 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 32m Warning Unhealthy pod/payload-processing-e2e-inf-b-10b00a-659db85879-m4pxv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc 32m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Container created 32m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Container started 32m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Container started 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-10b00a-5cdcfd8b5f-fwz5p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Add eth0 [10.134.0.218/23] from ovn-kubernetes 32m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Add eth0 [10.133.0.226/23] from ovn-kubernetes 32m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Container created 32m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Add eth0 [10.132.0.171/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Container created 32m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Container started 32m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Container started 32m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Container created 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Stopping container payload-pre-processing 32m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Startup probe error: Get "http://10.133.0.226:15021/healthz/ready": dial tcp 10.133.0.226:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Startup probe error: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Startup probe failed: Get "http://10.133.0.226:15021/healthz/ready": dial tcp 10.133.0.226:15021: connect: connection refused 32m Normal Killing pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Stopping container payload-processing 32m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Startup probe failed: Get "http://10.132.0.171:15021/healthz/ready": dial tcp 10.132.0.171:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Startup probe failed: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Startup probe error: Get "http://10.134.0.218:15021/healthz/ready": dial tcp 10.134.0.218:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Startup probe failed: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Startup probe error: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused... 32m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-rtv2k 32m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-rtv2k Stopping container discovery 32m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-f01de5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf Stopping container istio-proxy 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default New size: 3; reason: All metrics below target 32m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-f01de5-openshift-default Scaled up replica set e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f from 1 to 2 32m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-lb2mf 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default New size: 3; reason: All metrics below target 31m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 31m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-qcdd2 Stopping container istio-proxy 31m Normal Created pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Container created 31m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj Stopping container istio-proxy 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f Created pod: e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 31m Normal Started pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Container started 31m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-q2cv2 Stopping container istio-proxy 31m Normal AddedInterface pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Add eth0 [10.134.0.219/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-rmmkj 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning Unhealthy pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Startup probe failed: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Startup probe error: Get "http://10.134.0.219:15021/healthz/ready": dial tcp 10.134.0.219:15021: connect: connection refused... 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-a4240a-86f876f9cf-9bqvq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-processing-e2e-inf-a-a4240a-745976f465-st69v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Killing pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-a-a4240a-openshift-default-8449487f59-lmth8 Stopping container istio-proxy 31m Warning Unhealthy pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-inf-a-a4240a-openshift-default-8449487f59-zmwdc Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal AddedLabel gateway/e2e-shared-b-w1-c8537c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-c8537c 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-c8537c-openshift-default Scaled up replica set e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7 from 0 to 1 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7 Created pod: e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 31m Normal AddedInterface pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Add eth0 [10.134.0.220/23] from ovn-kubernetes 31m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf Created pod: payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 31m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-c8537c Scaled up replica set payload-processing-e2e-shared-b-w1-c8537c-8665fb9775 from 0 to 1 31m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775 Error creating: pods "payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-c8537c: serviceaccount "payload-processing-e2e-shared-b-w1-c8537c" not found 31m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-c8537c Scaled up replica set payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf from 0 to 1 31m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775 Created pod: payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 31m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf Error creating: pods "payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-c8537c: serviceaccount "payload-processing-e2e-shared-b-w1-c8537c" not found 31m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Container created 31m Normal Pulled pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Add eth0 [10.133.0.229/23] from ovn-kubernetes 31m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 31m Normal Created pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Container created 31m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Add eth0 [10.133.0.230/23] from ovn-kubernetes 31m Normal Started pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Container started 31m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Container started 31m Normal Killing pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-inf-b-0c091d-7d79b764b9-cnmxg Stopping container payload-processing 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default New size: 5; reason: All metrics below target 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-c78897-openshift-default Scaled up replica set e2e-shared-b-w2-c78897-openshift-default-7c9787498f from 4 to 5 31m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Container started 31m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Add eth0 [10.132.0.173/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Container created 31m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Startup probe failed: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Startup probe error: Get "http://10.132.0.173:15021/healthz/ready": dial tcp 10.132.0.173:15021: connect: connection refused... 31m Normal Killing pod/e2e-inf-b-0c091d-openshift-default-6855f44f99-srhdm Stopping container istio-proxy 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc Created pod: e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-7225e5-openshift-default Scaled up replica set e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc from 1 to 4 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc Created pod: e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc Created pod: e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-7225e5-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 31m Normal Pulled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Add eth0 [10.134.0.221/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Container created 31m Normal Started pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Container started 31m Normal AddedInterface pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Add eth0 [10.132.0.174/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Container created 31m Normal Started pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Container started 31m Normal Pulled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Add eth0 [10.133.0.233/23] from ovn-kubernetes 31m Normal Pulled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Created pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Container created 31m Normal Started pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Container started 31m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Container started 31m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Container created 31m Normal Started pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Container started 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-c8537c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal Created pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Container created 31m Normal Pulled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-c8537c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal Pulled pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-0c091d-7fb787cc8c-658b6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal Created pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Container created 31m Normal Started pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Container started 31m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Startup probe failed: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Startup probe error: Get "http://10.133.0.233:15021/healthz/ready": dial tcp 10.133.0.233:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Startup probe error: Get "http://10.134.0.221:15021/healthz/ready": dial tcp 10.134.0.221:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Startup probe failed: Get "http://10.134.0.221:15021/healthz/ready": dial tcp 10.134.0.221:15021: connect: connection refused 31m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Startup probe failed: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Startup probe error: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Startup probe failed: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Startup probe error: Get "http://10.132.0.119:15021/healthz/ready": dial tcp 10.132.0.119:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Startup probe failed: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Startup probe error: Get "http://10.134.0.217:15021/healthz/ready": dial tcp 10.134.0.217:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused... 31m Normal ScalingReplicaSet deployment/e2e-worker-w5-164db6-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 from 3 to 2 31m Normal SuccessfulDelete replicaset/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7 Deleted pod: e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-164db6-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 30m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-z4hmw Stopping container istio-proxy 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-7225e5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-7225e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-5ccd7b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-5ccd7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f Created pod: e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-5e2d03-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-5e2d03-openshift-default Scaled up replica set e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f from 1 to 2 30m Normal Pulled pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Container created 30m Normal Started pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Container started 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-9a482b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Normal AddedInterface pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Add eth0 [10.132.0.175/23] from ovn-kubernetes 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-9a482b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-7225e5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-7225e5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning Unhealthy pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 30m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-5ccd7b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-5ccd7b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9a482b-openshift-default Scaled up replica set e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 from 1 to 4 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Add eth0 [10.134.0.222/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Add eth0 [10.133.0.238/23] from ovn-kubernetes 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9a482b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Container created 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Container started 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Container created 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Container started 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Container created 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Add eth0 [10.134.0.223/23] from ovn-kubernetes 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Container started 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-7225e5-openshift-default New size: 5; reason: All metrics below target 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc Created pod: e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-7225e5-openshift-default Scaled up replica set e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc from 4 to 5 30m Normal Started pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Container started 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Startup probe error: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused... 30m Normal Pulled pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Container created 30m Normal AddedInterface pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Add eth0 [10.132.0.176/23] from ovn-kubernetes 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Startup probe failed: Get "http://10.133.0.238:15021/healthz/ready": dial tcp 10.133.0.238:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Startup probe error: Get "http://10.134.0.222:15021/healthz/ready": dial tcp 10.134.0.222:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Startup probe failed: Get "http://10.134.0.222:15021/healthz/ready": dial tcp 10.134.0.222:15021: connect: connection refused 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Startup probe failed: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Startup probe error: Get "http://10.134.0.223:15021/healthz/ready": dial tcp 10.134.0.223:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9a482b-openshift-default Scaled up replica set e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 from 4 to 8 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9a482b-openshift-default New size: 8; reason: 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Stopping container payload-pre-processing 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Add eth0 [10.134.0.224/23] from ovn-kubernetes 30m Normal Killing pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Stopping container payload-processing 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Add eth0 [10.133.0.240/23] from ovn-kubernetes 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Add eth0 [10.132.0.178/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Container started 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Container created 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Container started 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Container created 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Add eth0 [10.132.0.177/23] from ovn-kubernetes 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Container created 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Container started 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Container created 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Container started 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Startup probe error: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Startup probe failed: Get "http://10.133.0.240:15021/healthz/ready": dial tcp 10.133.0.240:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Startup probe error: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Startup probe failed: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Startup probe error: Get "http://10.132.0.178:15021/healthz/ready": dial tcp 10.132.0.178:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Startup probe failed: Get "http://10.132.0.177:15021/healthz/ready": dial tcp 10.132.0.177:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Startup probe error: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Startup probe failed: Get "http://10.134.0.224:15021/healthz/ready": dial tcp 10.134.0.224:15021: connect: connection refused 30m Warning ProbeError pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Readiness probe error: Get "http://10.134.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 30m Warning Unhealthy pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Readiness probe failed: Get "http://10.134.0.220:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 30m Normal Pulled pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Container started 30m Normal Created pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Container created 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 Created pod: e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-9a482b-openshift-default New size: 10; reason: 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Add eth0 [10.133.0.241/23] from ovn-kubernetes 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-9a482b-openshift-default Scaled up replica set e2e-shared-a-w4-9a482b-openshift-default-799986d6f9 from 8 to 10 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Container created 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Container started 30m Normal Started pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Container started 30m Normal Created pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Container created 30m Normal Pulled pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Add eth0 [10.134.0.226/23] from ovn-kubernetes 29m Normal SuccessfulCreate replicaset/e2e-inf-a-9e044c-openshift-default-85757fb4b9 Created pod: e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk 29m Normal Created pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Container created 29m Normal Pulled pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-9e044c-668c79d887-78jg4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal Started pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Container started 29m Normal AddedInterface pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Add eth0 [10.133.0.242/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-inf-a-9e044c-openshift-default Scaled up replica set e2e-inf-a-9e044c-openshift-default-85757fb4b9 from 2 to 3 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-9e044c-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Warning ProbeError pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Startup probe error: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Startup probe failed: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused 29m Warning Unhealthy pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Startup probe failed: Get "http://10.134.0.220:15021/healthz/ready": dial tcp 10.134.0.220:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Startup probe error: Get "http://10.133.0.241:15021/healthz/ready": dial tcp 10.133.0.241:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Startup probe failed: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Startup probe error: Get "http://10.134.0.226:15021/healthz/ready": dial tcp 10.134.0.226:15021: connect: connection refused... 29m Warning Unhealthy pod/payload-processing-e2e-inf-a-9e044c-75ffc66cdc-d5bnw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning ProbeError pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Startup probe error: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused... 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-734859-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal Killing pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-4zmbl Stopping container istio-proxy 29m Normal Killing pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Startup probe failed: Get "http://10.133.0.242:15021/healthz/ready": dial tcp 10.133.0.242:15021: connect: connection refused 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-734859-openshift-default Scaled up replica set e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f from 1 to 2 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f Created pod: e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv 29m Normal Started pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Container started 29m Normal Created pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Container created 29m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Container created 29m Normal Pulled pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Killing pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-tj5xk Stopping container istio-proxy 29m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Add eth0 [10.134.0.228/23] from ovn-kubernetes 29m Normal AddedInterface pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Add eth0 [10.134.0.227/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled up replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 2 to 3 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd 29m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Container started 29m Warning Unhealthy pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-inf-a-9e044c-openshift-default-85757fb4b9-jsnbv Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Startup probe failed: Get "http://10.134.0.228:15021/healthz/ready": dial tcp 10.134.0.228:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Startup probe error: Get "http://10.134.0.228:15021/healthz/ready": dial tcp 10.134.0.228:15021: connect: connection refused... 29m Warning ProbeError pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Startup probe error: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Startup probe failed: Get "http://10.134.0.227:15021/healthz/ready": dial tcp 10.134.0.227:15021: connect: connection refused 29m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default New size: 3; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-7r8ws Stopping container istio-proxy 29m Normal Killing pod/payload-processing-e2e-inf-b-ae79df-54c86dbff-2sfw2 Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Stopping container payload-pre-processing 29m Normal Killing pod/e2e-inf-b-ae79df-openshift-default-777549df6f-n7nfl Stopping container istio-proxy 29m Normal AddedLabel gateway/e2e-ipp-a-e1e630 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e1e630 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-e1e630-openshift-default-6576d59bc5 Created pod: e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn 29m Normal AddedInterface pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Add eth0 [10.134.0.229/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-ipp-a-e1e630-openshift-default Scaled up replica set e2e-ipp-a-e1e630-openshift-default-6576d59bc5 from 0 to 1 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-c8537c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-c8537c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-ae79df-bf79f4f59-rxs4s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e1e630-665899d59 Error creating: pods "payload-pre-processing-e2e-ipp-a-e1e630-665899d59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1e630: serviceaccount "payload-processing-e2e-ipp-a-e1e630" not found 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e1e630-7d69467b6f Error creating: pods "payload-processing-e2e-ipp-a-e1e630-7d69467b6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1e630: serviceaccount "payload-processing-e2e-ipp-a-e1e630" not found 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e1e630-7d69467b6f Created pod: payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e1e630-665899d59 Created pod: payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e1e630 Scaled up replica set payload-pre-processing-e2e-ipp-a-e1e630-665899d59 from 0 to 1 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e1e630 Scaled up replica set payload-processing-e2e-ipp-a-e1e630-7d69467b6f from 0 to 1 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm Container started 29m Normal Started pod/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj Container started 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm Container created 29m Normal Created pod/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj Container created 29m Normal Pulled pod/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm Add eth0 [10.133.0.245/23] from ovn-kubernetes 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj Add eth0 [10.133.0.244/23] from ovn-kubernetes 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-9f4dc9-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-9f4dc9-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 from 3 to 2 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5 Deleted pod: e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp 28m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-b9sqp Stopping container istio-proxy 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e1e630-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e1e630-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 28m Normal Killing pod/payload-processing-e2e-ipp-a-e1e630-7d69467b6f-fq7jj Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e1e630-665899d59-cmmgm Stopping container payload-pre-processing 28m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Stopping container istio-proxy 28m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 5 to 4 28m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-dh7c4 28m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e1e630-557c686fcd Error creating: pods "payload-processing-e2e-ipp-a-e1e630-557c686fcd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1e630: serviceaccount "payload-processing-e2e-ipp-a-e1e630" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e1e630-557c686fcd Created pod: payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e1e630 Scaled up replica set payload-processing-e2e-ipp-a-e1e630-557c686fcd from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79 Created pod: payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79 Error creating: pods "payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e1e630: serviceaccount "payload-processing-e2e-ipp-a-e1e630" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e1e630 Scaled up replica set payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79 from 0 to 1 28m Normal Pulling pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Add eth0 [10.133.0.247/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Add eth0 [10.133.0.248/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Container started 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.423s (2.423s including waiting). Image size: 166620835 bytes. 28m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Readiness probe error: Get "http://10.134.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-dh7c4 Readiness probe failed: Get "http://10.134.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Normal Pulled pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.463s (2.463s including waiting). Image size: 166620835 bytes. 28m Normal Created pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Container created 28m Normal Created pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Container created 28m Normal Started pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Container started 28m Normal Pulled pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning Unhealthy pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Startup probe failed: Get "http://10.134.0.229:15021/healthz/ready": dial tcp 10.134.0.229:15021: connect: connection refused 28m Warning ProbeError pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Startup probe error: Get "http://10.134.0.229:15021/healthz/ready": dial tcp 10.134.0.229:15021: connect: connection refused... 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-2b94b5-openshift-default-5b584bf549 Created pod: e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w 28m Normal AddedLabel gateway/e2e-ipp-b-2b94b5 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-2b94b5 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-2b94b5-openshift-default Scaled up replica set e2e-ipp-b-2b94b5-openshift-default-5b584bf549 from 0 to 1 28m Normal AddedInterface pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Add eth0 [10.134.0.230/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-2b94b5 Scaled up replica set payload-processing-e2e-ipp-b-2b94b5-758df97487 from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99 Error creating: pods "payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2b94b5: serviceaccount "payload-processing-e2e-ipp-b-2b94b5" not found 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-2b94b5-758df97487 Error creating: pods "payload-processing-e2e-ipp-b-2b94b5-758df97487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2b94b5: serviceaccount "payload-processing-e2e-ipp-b-2b94b5" not found 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-2b94b5 Scaled up replica set payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99 Created pod: payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-2b94b5-758df97487 Created pod: payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg Add eth0 [10.133.0.251/23] from ovn-kubernetes 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg Container started 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps Add eth0 [10.133.0.252/23] from ovn-kubernetes 28m Normal Created pod/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps Container created 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-104b6e-openshift-default Scaled up replica set e2e-ipp-a-104b6e-openshift-default-99f954b75 from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-a-104b6e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-104b6e 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-104b6e-openshift-default-99f954b75 Created pod: e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc 28m Normal AddedInterface pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Add eth0 [10.134.0.231/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-104b6e Scaled up replica set payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587 from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-104b6e Scaled up replica set payload-processing-e2e-ipp-a-104b6e-6dd5796b49 from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-104b6e-6dd5796b49 Created pod: payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-104b6e-6dd5796b49 Error creating: pods "payload-processing-e2e-ipp-a-104b6e-6dd5796b49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-104b6e: serviceaccount "payload-processing-e2e-ipp-a-104b6e" not found 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587 Error creating: pods "payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-104b6e: serviceaccount "payload-processing-e2e-ipp-a-104b6e" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587 Created pod: payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c Add eth0 [10.133.0.253/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg Container started 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg Add eth0 [10.133.0.254/23] from ovn-kubernetes 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e1e630-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e1e630-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-2b94b5-75f6495c99-c5ljg Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-ipp-b-2b94b5-758df97487-4d6ps Stopping container payload-processing 27m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-ipp-a-104b6e Failed to update endpoint openshift-ingress/payload-processing-e2e-ipp-a-104b6e: Operation cannot be fulfilled on endpoints "payload-processing-e2e-ipp-a-104b6e": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-ipp-a-104b6e, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 3c81c3cc-c665-45f2-a6ce-1c8b8b8937a7, UID in object meta: 27m Normal Killing pod/payload-processing-e2e-ipp-a-104b6e-6dd5796b49-86v9c Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-ipp-a-104b6e-6dfb84c587-8rgrg Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-2b94b5 Scaled up replica set payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc from 0 to 1 27m Normal Created pod/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l Container created 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-2b94b5 Scaled up replica set payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb from 0 to 1 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc Error creating: pods "payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2b94b5: serviceaccount "payload-processing-e2e-ipp-b-2b94b5" not found 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l Add eth0 [10.133.1.3/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc Created pod: payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Add eth0 [10.132.0.179/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb Created pod: payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l 27m Normal Pulled pod/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 27m Normal Started pod/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l Container started 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb Error creating: pods "payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-2b94b5: serviceaccount "payload-processing-e2e-ipp-b-2b94b5" not found 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Container started 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.458s (1.458s including waiting). Image size: 166620835 bytes. 27m Normal SuccessfulDelete replicaset/e2e-worker-w1-dfff59-openshift-default-5fd958c56c Deleted pod: e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf 27m Normal ScalingReplicaSet deployment/e2e-worker-w1-dfff59-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-dfff59-openshift-default-5fd958c56c from 3 to 2 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-dfff59-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 27m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Stopping container istio-proxy 27m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Readiness probe error: HTTP probe failed with statuscode: 503... 27m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5hnrf Readiness probe failed: HTTP probe failed with statuscode: 503 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz Add eth0 [10.133.1.4/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-104b6e Scaled up replica set payload-processing-e2e-ipp-a-104b6e-776769677d from 0 to 1 27m Normal Pulled pod/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-104b6e-776769677d Created pod: payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-104b6e-776769677d Error creating: pods "payload-processing-e2e-ipp-a-104b6e-776769677d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-104b6e: serviceaccount "payload-processing-e2e-ipp-a-104b6e" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59 Created pod: payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59 Error creating: pods "payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-104b6e: serviceaccount "payload-processing-e2e-ipp-a-104b6e" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-104b6e Scaled up replica set payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59 from 0 to 1 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Add eth0 [10.134.0.232/23] from ovn-kubernetes 27m Normal Started pod/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz Container created 27m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Container started 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.192s (2.192s including waiting). Image size: 166620835 bytes. 26m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-6b0e0f-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/e2e-worker-w2-6b0e0f-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-6b0e0f-openshift-default-7966746dbc from 3 to 2 26m Normal SuccessfulDelete replicaset/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc Deleted pod: e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j 26m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-tnj4j Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-bd831a-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/e2e-worker-w3-bd831a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-bd831a-openshift-default-5b796d5fbf from 3 to 2 26m Normal SuccessfulDelete replicaset/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf Deleted pod: e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr 26m Normal AddedLabel gateway/e2e-ipp-b-7321d6 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-7321d6 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-7321d6-openshift-default Scaled up replica set e2e-ipp-b-7321d6-openshift-default-65fdb447 from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-7321d6-openshift-default-65fdb447 Created pod: e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq 26m Warning Unhealthy pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Readiness probe failed: HTTP probe failed with statuscode: 503 26m Warning ProbeError pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-vr5hr Readiness probe error: HTTP probe failed with statuscode: 503... 26m Normal AddedInterface pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Add eth0 [10.134.0.234/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47 Error creating: pods "payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7321d6: serviceaccount "payload-processing-e2e-ipp-b-7321d6" not found 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-7321d6 Scaled up replica set payload-processing-e2e-ipp-b-7321d6-55df99b949 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47 Created pod: payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-7321d6 Scaled up replica set payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-7321d6-55df99b949 Error creating: pods "payload-processing-e2e-ipp-b-7321d6-55df99b949-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7321d6: serviceaccount "payload-processing-e2e-ipp-b-7321d6" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-7321d6-55df99b949 Created pod: payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 Add eth0 [10.134.0.235/23] from ovn-kubernetes 26m Normal Started pod/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 Container started 26m Normal Created pod/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc Add eth0 [10.134.0.236/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Pulled pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Container started 26m Normal Created pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Container created 26m Normal Started pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Container started 26m Normal Created pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Container created 26m Normal Pulled pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning Unhealthy pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Startup probe failed: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Startup probe error: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused... 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-c78897-openshift-default Scaled up replica set e2e-shared-b-w2-c78897-openshift-default-7c9787498f from 5 to 7 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 26m Warning ProbeError pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Startup probe error: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused... 26m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Container started 26m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Container created 26m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Container created 26m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Add eth0 [10.133.1.9/23] from ovn-kubernetes 26m Warning Unhealthy pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Startup probe failed: Get "http://10.134.0.230:15021/healthz/ready": dial tcp 10.134.0.230:15021: connect: connection refused 26m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Add eth0 [10.132.0.182/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Container started 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-104b6e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-104b6e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Startup probe error: Get "http://10.133.1.9:15021/healthz/ready": dial tcp 10.133.1.9:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Startup probe failed: Get "http://10.133.1.9:15021/healthz/ready": dial tcp 10.133.1.9:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Startup probe error: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Startup probe failed: Get "http://10.132.0.182:15021/healthz/ready": dial tcp 10.132.0.182:15021: connect: connection refused 26m Normal SuccessfulDelete replicaset/e2e-worker-w0-4a083d-openshift-default-888567bc7 Deleted pod: e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn 26m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-zrljn Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-4a083d-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/e2e-worker-w0-4a083d-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-4a083d-openshift-default-888567bc7 from 3 to 2 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-7321d6-64bfb77c47-v7d22 Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-7321d6-55df99b949-vs6jc Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Stopping container payload-processing 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-43fac8-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 25m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-worker-w6-43fac8-openshift-default-84b4466d8c Deleted pod: e2e-worker-w6-43fac8-openshift-default-84b4466d8c-t799w 25m Normal ScalingReplicaSet deployment/e2e-worker-w6-43fac8-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-43fac8-openshift-default-84b4466d8c from 3 to 2 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-7321d6 Scaled up replica set payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-7321d6 Scaled up replica set payload-processing-e2e-ipp-b-7321d6-5c5749c946 from 0 to 1 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-104b6e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7321d6: serviceaccount "payload-processing-e2e-ipp-b-7321d6" not found 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-104b6e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9 Created pod: payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-7321d6-5c5749c946 Created pod: payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-7321d6-5c5749c946 Error creating: pods "payload-processing-e2e-ipp-b-7321d6-5c5749c946-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-7321d6: serviceaccount "payload-processing-e2e-ipp-b-7321d6" not found 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-2b94b5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-2b94b5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg Add eth0 [10.134.0.243/23] from ovn-kubernetes 25m Normal Created pod/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z Container created 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z Add eth0 [10.134.0.242/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z Container started 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal Pulled pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Killing pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-n72bg Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Stopping container istio-proxy 25m Normal Created pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Container created 25m Normal Started pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Container started 25m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-7225e5-d8c8dcb65-ts6np Readiness probe failed: timeout: failed to connect service "10.134.0.207:50052" within 1s: context deadline exceeded 25m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-qlx2l Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-mnzch Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning Unhealthy pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-q495c Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-shared-a-w2-7225e5-openshift-default-59ccbb9cdc-cvmjn Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-7225e5-db455946c-psv2h Readiness probe failed: timeout: failed to connect service "10.134.0.206:50052" within 1s: context deadline exceeded 25m Warning Unhealthy pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Startup probe failed: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Startup probe error: Get "http://10.134.0.234:15021/healthz/ready": dial tcp 10.134.0.234:15021: connect: connection refused... 25m Normal Killing pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Stopping container payload-pre-processing 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 25m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Container started 25m Normal Started pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Container started 25m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Container created 25m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled up replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 2 to 4 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-c78897-openshift-default Scaled up replica set e2e-shared-b-w2-c78897-openshift-default-7c9787498f from 7 to 9 25m Normal Created pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Container created 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-c78897-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr 25m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Add eth0 [10.133.1.11/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-c78897-openshift-default-7c9787498f Created pod: e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal AddedInterface pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Add eth0 [10.134.0.249/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p 25m Normal Pulled pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Container started 25m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Container created 25m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Container started 25m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Add eth0 [10.133.1.12/23] from ovn-kubernetes 25m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Container created 25m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Add eth0 [10.134.0.250/23] from ovn-kubernetes 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-7321d6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-7321d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 Created pod: e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f Created pod: e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m 25m Normal AddedInterface pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Add eth0 [10.132.0.184/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-734859-openshift-default Scaled up replica set e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f from 2 to 3 25m Normal Pulled pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-734859-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-f47890-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-f47890-openshift-default Scaled up replica set e2e-shared-a-w3-f47890-openshift-default-557cbcdc57 from 3 to 4 25m Normal Pulled pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal Started pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Container started 25m Normal Created pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Container created 25m Normal Created pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Container created 25m Normal Started pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Container started 25m Normal AddedInterface pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Add eth0 [10.134.0.251/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 Created pod: e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-383fde-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-383fde-openshift-default Scaled up replica set e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 from 2 to 3 25m Normal Created pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Container created 25m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Startup probe failed: Get "http://10.133.1.11:15021/healthz/ready": dial tcp 10.133.1.11:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Startup probe error: Get "http://10.133.1.11:15021/healthz/ready": dial tcp 10.133.1.11:15021: connect: connection refused... 25m Normal Started pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Container started 25m Normal Pulled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Add eth0 [10.132.0.185/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Startup probe error: Get "http://10.133.1.12:15021/healthz/ready": dial tcp 10.133.1.12:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Startup probe failed: Get "http://10.134.0.249:15021/healthz/ready": dial tcp 10.134.0.249:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Startup probe error: Get "http://10.134.0.249:15021/healthz/ready": dial tcp 10.134.0.249:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Startup probe failed: Get "http://10.133.1.12:15021/healthz/ready": dial tcp 10.133.1.12:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Startup probe failed: Get "http://10.134.0.250:15021/healthz/ready": dial tcp 10.134.0.250:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Startup probe error: Get "http://10.134.0.250:15021/healthz/ready": dial tcp 10.134.0.250:15021: connect: connection refused... 25m Warning ProbeError pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Startup probe error: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Startup probe failed: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused 25m Warning Unhealthy pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Startup probe failed: Get "http://10.134.0.251:15021/healthz/ready": dial tcp 10.134.0.251:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Startup probe error: Get "http://10.134.0.251:15021/healthz/ready": dial tcp 10.134.0.251:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 25m Warning ProbeError pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-bvlp7 Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-cdgzc Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-79q4s Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-gxmtr Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-78zkt Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-5rmp5 Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-rc28h Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-v576x Stopping container istio-proxy 25m Normal Killing pod/e2e-shared-b-w2-c78897-openshift-default-7c9787498f-kx72b Stopping container istio-proxy 25m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-ipp-a-104b6e Failed to update endpoint openshift-ingress/payload-processing-e2e-ipp-a-104b6e: Operation cannot be fulfilled on endpoints "payload-processing-e2e-ipp-a-104b6e": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-ipp-a-104b6e, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: cae3d727-1cad-4f73-9b79-1dc21d80d440, UID in object meta: 25m Normal Killing pod/payload-processing-e2e-ipp-a-104b6e-776769677d-8h8mz Stopping container payload-processing 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-7321d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-104b6e-675cc9cb59-vmxp4 Stopping container payload-pre-processing 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-c78897-86d6f6596c-74kkb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-7321d6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-c78897-68dd595655-bgrvt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 25m Normal AddedLabel gateway/e2e-ipp-a-3fd73f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-3fd73f 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-3fd73f-openshift-default Scaled up replica set e2e-ipp-a-3fd73f-openshift-default-5d5658744f from 0 to 1 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-3fd73f-openshift-default-5d5658744f Created pod: e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr 25m Normal AddedInterface pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Add eth0 [10.134.0.253/23] from ovn-kubernetes 25m Normal Killing pod/e2e-ipp-a-104b6e-openshift-default-99f954b75-nrzqc Stopping container istio-proxy 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-3fd73f-596f898c95 Created pod: payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-3fd73f Scaled up replica set payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99 Created pod: payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-3fd73f Scaled up replica set payload-processing-e2e-ipp-a-3fd73f-596f898c95 from 0 to 1 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99 Error creating: pods "payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3fd73f: serviceaccount "payload-processing-e2e-ipp-a-3fd73f" not found 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-3fd73f-596f898c95 Error creating: pods "payload-processing-e2e-ipp-a-3fd73f-596f898c95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3fd73f: serviceaccount "payload-processing-e2e-ipp-a-3fd73f" not found 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 Add eth0 [10.133.1.14/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl Add eth0 [10.133.1.15/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Created pod/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 Container created 25m Normal Started pod/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl Container started 25m Normal Killing pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Stopping container payload-processing 24m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-6b0e0f-765874f899-9752p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-processing-e2e-worker-w2-6b0e0f-7bc6c99cbf-scggh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Normal Killing pod/payload-processing-e2e-ipp-b-7321d6-5c5749c946-tmb8z Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-7321d6-6f78d745c9-xgnqg Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-3fd73f-596f898c95-xsw58 Stopping container payload-processing 24m Normal Killing pod/e2e-ipp-b-7321d6-openshift-default-65fdb447-gv9jq Stopping container istio-proxy 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-3fd73f-745bf87d99-rh4hl Stopping container payload-pre-processing 24m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-65hkq Stopping container istio-proxy 24m Normal Killing pod/e2e-worker-w2-6b0e0f-openshift-default-7966746dbc-87srx Stopping container istio-proxy 24m Normal Killing pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Stopping container payload-pre-processing 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-028c46-openshift-default-5c6465b9f7 Created pod: e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl 24m Normal AddedLabel gateway/e2e-ipp-a-028c46 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-028c46 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-028c46-openshift-default Scaled up replica set e2e-ipp-a-028c46-openshift-default-5c6465b9f7 from 0 to 1 24m Normal AddedInterface pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Add eth0 [10.134.1.0/23] from ovn-kubernetes 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c Error creating: pods "payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3fd73f: serviceaccount "payload-processing-e2e-ipp-a-3fd73f" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-3fd73f Scaled up replica set payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-3fd73f Scaled up replica set payload-processing-e2e-ipp-a-3fd73f-9c5644f64 from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-3fd73f-9c5644f64 Error creating: pods "payload-processing-e2e-ipp-a-3fd73f-9c5644f64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-3fd73f: serviceaccount "payload-processing-e2e-ipp-a-3fd73f" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c Created pod: payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-028c46-568c99d987 Error creating: pods "payload-pre-processing-e2e-ipp-a-028c46-568c99d987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-028c46: serviceaccount "payload-processing-e2e-ipp-a-028c46" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-028c46 Scaled up replica set payload-processing-e2e-ipp-a-028c46-675df4f67f from 0 to 1 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw Add eth0 [10.133.1.21/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw Container created 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw Container started 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-028c46-675df4f67f Created pod: payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-028c46 Scaled up replica set payload-pre-processing-e2e-ipp-a-028c46-568c99d987 from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-028c46-675df4f67f Error creating: pods "payload-processing-e2e-ipp-a-028c46-675df4f67f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-028c46: serviceaccount "payload-processing-e2e-ipp-a-028c46" not found 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv Add eth0 [10.133.1.22/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv Container created 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 Add eth0 [10.134.1.1/23] from ovn-kubernetes 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-3fd73f-9c5644f64 Created pod: payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-028c46-568c99d987 Created pod: payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c Add eth0 [10.133.1.23/23] from ovn-kubernetes 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv Container started 24m Normal Started pod/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 Container started 24m Normal Created pod/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c Container started 24m Normal Pulled pod/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c Container created 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-5e2d03-757556fddc-j9gph Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-5e2d03-cbd9bb6f9-bbprx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-6b6cfc-openshift-default Scaled up replica set e2e-ipp-b-6b6cfc-openshift-default-75cc4f89 from 0 to 1 23m Normal AddedLabel gateway/e2e-ipp-b-6b6cfc Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-6b6cfc 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89 Created pod: e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 23m Normal AddedInterface pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Add eth0 [10.134.1.11/23] from ovn-kubernetes 23m Normal Killing pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-vzvrz Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w0-5e2d03-openshift-default-577cbdbb4f-tt9zz Stopping container istio-proxy 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x Add eth0 [10.133.1.33/23] from ovn-kubernetes 23m Normal Created pod/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x Container created 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-6b6cfc Scaled up replica set payload-processing-e2e-ipp-b-6b6cfc-7f499968b7 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc Created pod: payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc Error creating: pods "payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6b6cfc: serviceaccount "payload-processing-e2e-ipp-b-6b6cfc" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7 Created pod: payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x 23m Normal Pulled pod/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-6b6cfc Scaled up replica set payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc from 0 to 1 23m Normal Started pod/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x Container started 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7 Error creating: pods "payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6b6cfc: serviceaccount "payload-processing-e2e-ipp-b-6b6cfc" not found 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 Add eth0 [10.134.1.13/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 Container started 23m Normal Killing pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Stopping container payload-pre-processing 23m Normal Pulled pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Container created 23m Normal Started pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Container started 23m Normal Killing pod/payload-processing-e2e-ipp-a-028c46-675df4f67f-8l65c Stopping container payload-processing 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-3fd73f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-3fd73f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-028c46-568c99d987-m48cv Stopping container payload-pre-processing 23m Warning ProbeError pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Startup probe error: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Startup probe failed: Get "http://10.134.0.253:15021/healthz/ready": dial tcp 10.134.0.253:15021: connect: connection refused 23m Warning FailedMount pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 MountVolume.SetUp failed for volume "plugins-config-volume" : configmap "payload-processing-plugins-e2e-shared-b-w0-c8d121" not found 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e1e630-5b5f67bc79-j5ltm Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Stopping container payload-processing 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-028c46 Scaled up replica set payload-pre-processing-e2e-ipp-a-028c46-769655699f from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-028c46 Scaled up replica set payload-processing-e2e-ipp-a-028c46-9f59dd876 from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-028c46-9f59dd876 Error creating: pods "payload-processing-e2e-ipp-a-028c46-9f59dd876-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-028c46: serviceaccount "payload-processing-e2e-ipp-a-028c46" not found 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-028c46-769655699f Error creating: pods "payload-pre-processing-e2e-ipp-a-028c46-769655699f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-028c46: serviceaccount "payload-processing-e2e-ipp-a-028c46" not found 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq Container started 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-028c46-769655699f Created pod: payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Killing pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-ncnzv Stopping container istio-proxy 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq Container created 23m Normal Killing pod/e2e-shared-b-w0-c8d121-openshift-default-786dfd7b4c-jhtvl Stopping container istio-proxy 23m Warning Unhealthy pod/payload-processing-e2e-ipp-a-e1e630-557c686fcd-w6sbm Readiness probe failed: dial tcp 10.133.0.248:9004: connect: connection refused 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq Add eth0 [10.133.1.34/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-028c46-9f59dd876 Created pod: payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr Add eth0 [10.134.1.15/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr Container started 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-c8d121-5595856559-g7tk9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-c8d121-66bb6ff85-8mssn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-028c46-openshift-default-5c6465b9f7 Created pod: e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r 23m Normal ScalingReplicaSet deployment/e2e-ipp-a-028c46-openshift-default Scaled up replica set e2e-ipp-a-028c46-openshift-default-5c6465b9f7 from 1 to 4 23m Normal AddedInterface pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Add eth0 [10.134.1.16/23] from ovn-kubernetes 23m Normal Pulled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-028c46-openshift-default-5c6465b9f7 Created pod: e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r 23m Normal Pulled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal SuccessfulCreate replicaset/e2e-ipp-a-028c46-openshift-default-5c6465b9f7 Created pod: e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 23m Normal AddedInterface pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Add eth0 [10.133.1.36/23] from ovn-kubernetes 23m Normal AddedInterface pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Add eth0 [10.132.0.190/23] from ovn-kubernetes 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-028c46-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 23m Normal Pulled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Container created 23m Normal Started pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Container started 23m Normal Killing pod/payload-processing-e2e-ipp-b-6b6cfc-7f499968b7-xdg7x Stopping container payload-processing 23m Normal Created pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Container created 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-6b6cfc-6f8b7f8fdc-qfwn9 Stopping container payload-pre-processing 23m Normal Created pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Container created 23m Normal Started pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Container started 23m Normal Started pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Container started 23m Warning Unhealthy pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Readiness probe failed: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Normal Killing pod/payload-pre-processing-e2e-worker-w0-4a083d-cdfc4788c-48knm Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-worker-w0-4a083d-84c576c9b7-jlscm Stopping container payload-processing 23m Warning ProbeError pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Readiness probe error: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Warning ProbeError pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Startup probe error: Get "http://10.132.0.190:15021/healthz/ready": dial tcp 10.132.0.190:15021: connect: connection refused... 23m Warning ProbeError pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Startup probe error: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Startup probe failed: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused 23m Normal Created pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Container created 23m Warning Unhealthy pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Startup probe failed: Get "http://10.134.1.16:15021/healthz/ready": dial tcp 10.134.1.16:15021: connect: connection refused 23m Warning Unhealthy pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Startup probe failed: Get "http://10.132.0.190:15021/healthz/ready": dial tcp 10.132.0.190:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Startup probe error: Get "http://10.133.1.36:15021/healthz/ready": dial tcp 10.133.1.36:15021: connect: connection refused... 23m Normal Started pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Container started 23m Normal Pulled pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-3fd73f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-3fd73f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal AddedLabel gateway/e2e-ipp-b-aa6303 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-aa6303 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85 Created pod: e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-aa6303-openshift-default Scaled up replica set e2e-ipp-b-aa6303-openshift-default-6cc7c47b85 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-6b6cfc Scaled up replica set payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75 from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-6b6cfc-67879db544 Error creating: pods "payload-processing-e2e-ipp-b-6b6cfc-67879db544-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6b6cfc: serviceaccount "payload-processing-e2e-ipp-b-6b6cfc" not found 23m Warning ProbeError pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Startup probe error: Get "http://10.134.1.0:15021/healthz/ready": dial tcp 10.134.1.0:15021: connect: connection refused... 23m Normal AddedInterface pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Add eth0 [10.134.1.18/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-6b6cfc-67879db544 Created pod: payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75 Created pod: payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx 23m Warning Unhealthy pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Startup probe failed: Get "http://10.134.1.0:15021/healthz/ready": dial tcp 10.134.1.0:15021: connect: connection refused 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-6b6cfc Scaled up replica set payload-processing-e2e-ipp-b-6b6cfc-67879db544 from 0 to 1 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75 Error creating: pods "payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-6b6cfc: serviceaccount "payload-processing-e2e-ipp-b-6b6cfc" not found 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx Add eth0 [10.134.1.19/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv Container started 23m Normal Created pod/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv Container created 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv Add eth0 [10.133.1.38/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx Container created 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx Container started 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-028c46-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-028c46-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-aa6303 Scaled up replica set payload-pre-processing-e2e-ipp-b-aa6303-667996645f from 0 to 1 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5 Error creating: pods "payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa6303: serviceaccount "payload-processing-e2e-ipp-b-aa6303" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-aa6303 Scaled up replica set payload-processing-e2e-ipp-b-aa6303-59d6d99bc5 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-aa6303-667996645f Created pod: payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-aa6303-667996645f Error creating: pods "payload-pre-processing-e2e-ipp-b-aa6303-667996645f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa6303: serviceaccount "payload-processing-e2e-ipp-b-aa6303" not found 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5 Created pod: payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 Add eth0 [10.133.1.39/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Pulled pod/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 Container created 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz Add eth0 [10.133.1.40/23] from ovn-kubernetes 22m Normal Started pod/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 Container started 22m Normal Started pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Container started 22m Normal Created pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Container created 22m Normal Pulled pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Stopping container istio-proxy 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6b6cfc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6b6cfc-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Killing pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Stopping container istio-proxy 22m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning ProbeError pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-bk68b Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning Unhealthy pod/e2e-worker-w0-4a083d-openshift-default-888567bc7-z5wwj Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Startup probe error: Get "http://10.134.1.11:15021/healthz/ready": dial tcp 10.134.1.11:15021: connect: connection refused... 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-3fd73f-6f5d95576c-r9lrw Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-3fd73f-9c5644f64-xrw75 Stopping container payload-processing 22m Warning Unhealthy pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Startup probe failed: Get "http://10.134.1.11:15021/healthz/ready": dial tcp 10.134.1.11:15021: connect: connection refused 22m Normal Killing pod/e2e-ipp-a-e1e630-openshift-default-6576d59bc5-ggbwn Stopping container istio-proxy 22m Normal Killing pod/e2e-ipp-a-3fd73f-openshift-default-5d5658744f-zdxwr Stopping container istio-proxy 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-aa6303-667996645f-xzklz Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-aa6303-59d6d99bc5-lqbj5 Stopping container payload-processing 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-aa6303-5bcf66496 Error creating: pods "payload-processing-e2e-ipp-b-aa6303-5bcf66496-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa6303: serviceaccount "payload-processing-e2e-ipp-b-aa6303" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-aa6303 Scaled up replica set payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f Created pod: payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f Error creating: pods "payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-aa6303: serviceaccount "payload-processing-e2e-ipp-b-aa6303" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-aa6303 Scaled up replica set payload-processing-e2e-ipp-b-aa6303-5bcf66496 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-aa6303-5bcf66496 Created pod: payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq Add eth0 [10.134.1.22/23] from ovn-kubernetes 22m Normal Pulled pod/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw Add eth0 [10.134.1.21/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw Container started 22m Normal Started pod/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw Container created 22m Normal Created pod/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq Container created 22m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-b-2b94b5 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-2b94b5: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-2b94b5": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-2b94b5, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: f657f604-879b-4202-8371-e00645de0efa, UID in object meta: 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-2b94b5-56955c97dc-rmhx8 Stopping container payload-pre-processing 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-6b6cfc-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-6b6cfc-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal Killing pod/payload-processing-e2e-ipp-b-2b94b5-75d8b89bfb-7kk7l Stopping container payload-processing 22m Normal Created pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Container created 22m Normal Started pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Container started 22m Normal Pulled pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-6b6cfc-779fd7cf75-tnrpx Stopping container payload-pre-processing 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-aa6303-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-aa6303-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Normal Killing pod/payload-processing-e2e-ipp-b-6b6cfc-67879db544-brnzv Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-b-2b94b5-openshift-default-5b584bf549-h5b5w Stopping container istio-proxy 22m Warning ProbeError pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Startup probe error: Get "http://10.134.1.18:15021/healthz/ready": dial tcp 10.134.1.18:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Startup probe failed: Get "http://10.134.1.18:15021/healthz/ready": dial tcp 10.134.1.18:15021: connect: connection refused 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-a9a967-openshift-default Scaled up replica set e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 from 4 to 5 21m Normal Killing pod/e2e-ipp-b-6b6cfc-openshift-default-75cc4f89-d6dw4 Stopping container istio-proxy 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-a9a967-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5 Created pod: e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 21m Normal Pulled pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Container started 21m Normal AddedInterface pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Add eth0 [10.133.1.47/23] from ovn-kubernetes 21m Normal Created pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Container created 21m Normal Killing pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Stopping container payload-pre-processing 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-383fde-openshift-default Scaled up replica set e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 from 3 to 4 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w5-383fde-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 21m Normal Started pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Container started 21m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59 Created pod: e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr 21m Normal Created pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Container created 21m Normal Killing pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Stopping container payload-processing 21m Normal Pulled pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Add eth0 [10.134.1.25/23] from ovn-kubernetes 21m Warning ProbeError pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Startup probe error: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Startup probe failed: Get "http://10.133.1.47:15021/healthz/ready": dial tcp 10.133.1.47:15021: connect: connection refused 21m Normal Killing pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Stopping container payload-pre-processing 21m Warning Unhealthy pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Startup probe failed: Get "http://10.134.1.25:15021/healthz/ready": dial tcp 10.134.1.25:15021: connect: connection refused 21m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-028c46 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-028c46: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-028c46": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-028c46, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 0751e9ea-ea80-408c-90ff-fdb9d580f977, UID in object meta: 21m Warning ProbeError pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Startup probe error: Get "http://10.134.1.25:15021/healthz/ready": dial tcp 10.134.1.25:15021: connect: connection refused... 21m Normal Killing pod/payload-processing-e2e-ipp-a-028c46-9f59dd876-9bxwr Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-028c46-769655699f-kzrnq Stopping container payload-pre-processing 21m Normal Killing pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-vzqz2 Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-bh45r Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-p8v9r Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-028c46-openshift-default-5c6465b9f7-ckkbl Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-43fac8-84b47867b7-4gzrb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/payload-processing-e2e-worker-w6-43fac8-86d989f9c-jnlzs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-h5vn9 Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-bbrlr Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cfhrd Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w3-f47890-openshift-default-557cbcdc57-cjpll Stopping container istio-proxy 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-f47890-b7b766d4c-bm926 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal ScalingReplicaSet deployment/e2e-ipp-a-501d63-openshift-default Scaled up replica set e2e-ipp-a-501d63-openshift-default-b85b799f7 from 0 to 1 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-7f5667f86c from 4 to 3 21m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-7f5667f86c Deleted pod: maas-default-gateway-openshift-default-7f5667f86c-hfzvl 21m Normal AddedLabel gateway/e2e-ipp-a-501d63 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-501d63 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-501d63-openshift-default-b85b799f7 Created pod: e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 21m Normal Killing pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Stopping container istio-proxy 21m Normal AddedInterface pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Add eth0 [10.134.1.28/23] from ovn-kubernetes 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-f47890-84fdfb9c59-fs5nn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-p5ffs Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-worker-w6-43fac8-openshift-default-84b4466d8c-m7cfl Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-501d63-d548b6d79 Error creating: pods "payload-processing-e2e-ipp-a-501d63-d548b6d79-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-501d63: serviceaccount "payload-processing-e2e-ipp-a-501d63" not found 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-501d63-6689474455 Error creating: pods "payload-pre-processing-e2e-ipp-a-501d63-6689474455-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-501d63: serviceaccount "payload-processing-e2e-ipp-a-501d63" not found 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-501d63-d548b6d79 Created pod: payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-501d63-6689474455 Created pod: payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-501d63 Scaled up replica set payload-pre-processing-e2e-ipp-a-501d63-6689474455 from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-501d63 Scaled up replica set payload-processing-e2e-ipp-a-501d63-d548b6d79 from 0 to 1 21m Normal Pulled pod/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-aa6303-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Readiness probe failed: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-aa6303-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal Started pod/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd Container started 21m Normal Created pod/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd Container created 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd Add eth0 [10.134.1.29/23] from ovn-kubernetes 21m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hfzvl Readiness probe error: Get "http://10.132.0.55:15021/healthz/ready": dial tcp 10.132.0.55:15021: connect: connection refused... 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q Container created 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q Container started 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q Add eth0 [10.134.1.30/23] from ovn-kubernetes 21m Normal Killing pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Stopping container payload-pre-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-aa6303-cd5bc765f-9f5tw Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-aa6303-5bcf66496-8dckq Stopping container payload-processing 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-f01de5-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 21m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f Created pod: e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-f01de5-openshift-default Scaled up replica set e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f from 2 to 3 21m Normal Started pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Container started 21m Normal AddedInterface pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Add eth0 [10.132.0.191/23] from ovn-kubernetes 21m Normal Pulled pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Container created 21m Normal Killing pod/e2e-ipp-b-aa6303-openshift-default-6cc7c47b85-ccx5h Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 21m Warning ProbeError pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 21m Normal Killing pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning Unhealthy pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-lkr5m Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning ProbeError pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-fxjdv Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning ProbeError pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w3-734859-openshift-default-d8d8ddd9f-nr6df Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-a-501d63-d548b6d79-w9jxd Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-501d63-6689474455-mhl6q Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Stopping container payload-processing 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-734859-5fc5fb544f-n2j8r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-734859-589cbc787-xqhjc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-501d63 Scaled up replica set payload-pre-processing-e2e-ipp-a-501d63-678bbff655 from 0 to 1 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb Error creating: pods "payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-501d63: serviceaccount "payload-processing-e2e-ipp-a-501d63" not found 20m Normal Killing pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Stopping container payload-processing 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-501d63 Scaled up replica set payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-501d63-678bbff655 Error creating: pods "payload-pre-processing-e2e-ipp-a-501d63-678bbff655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-501d63: serviceaccount "payload-processing-e2e-ipp-a-501d63" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-501d63-678bbff655 Created pod: payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s 20m Normal Killing pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Stopping container payload-pre-processing 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Pulled pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Add eth0 [10.134.1.33/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb Created pod: payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Add eth0 [10.134.1.32/23] from ovn-kubernetes 20m Normal Started pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Container started 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Container started 20m Normal Created pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Container created 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Container created 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-5ccd7b-655967d999-p8xqs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-a-w1-5ccd7b-openshift-default-766649b8f-rg2z4 Stopping container istio-proxy 20m Normal Created pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Container created 20m Normal Started pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Container started 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-501d63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-501d63-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Pulled pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-79lv7 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-sxzwq Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-2f4vt Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-x5c4j Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-wgfxf Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-r5gbg Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-jn747 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-9grjv Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-cr524 Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Startup probe failed: Get "http://10.134.1.28:15021/healthz/ready": dial tcp 10.134.1.28:15021: connect: connection refused 20m Normal AddedLabel gateway/e2e-ipp-b-f4a265 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-f4a265 20m Warning ProbeError pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Startup probe error: Get "http://10.134.1.28:15021/healthz/ready": dial tcp 10.134.1.28:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-a-w4-9a482b-openshift-default-799986d6f9-6g9zz Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-f4a265-openshift-default-544bb4948f Created pod: e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-f4a265-openshift-default Scaled up replica set e2e-ipp-b-f4a265-openshift-default-544bb4948f from 0 to 1 20m Normal AddedInterface pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Add eth0 [10.134.1.34/23] from ovn-kubernetes 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-5ccd7b-6f9b864d6c-2dft4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-9a482b-86b559b58f-frnp7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-9a482b-79b5ff7995-5zhdr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Stopping container payload-pre-processing 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f Error creating: pods "payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f4a265: serviceaccount "payload-processing-e2e-ipp-b-f4a265" not found 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f4a265 Scaled up replica set payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f4a265 Scaled up replica set payload-processing-e2e-ipp-b-f4a265-5db88cf775 from 0 to 1 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f4a265-5db88cf775 Error creating: pods "payload-processing-e2e-ipp-b-f4a265-5db88cf775-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f4a265: serviceaccount "payload-processing-e2e-ipp-b-f4a265" not found 20m Normal Killing pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Stopping container payload-processing 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc Add eth0 [10.133.1.61/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f Created pod: payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f4a265-5db88cf775 Created pod: payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc Container started 20m Normal Started pod/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 Container started 20m Normal Created pod/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 Container created 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 Add eth0 [10.134.1.35/23] from ovn-kubernetes 20m Normal Killing pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-processing-e2e-worker-w3-bd831a-544dc6b955-mn6k9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-bd831a-56899656f5-jhvtb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-5wdfs Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w3-bd831a-openshift-default-5b796d5fbf-bnmxh Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w1-c8537c-openshift-default-5bb45fc5c7-fbfw4 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-qtrd5 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-bqnc7 Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-c8537c-dcfd4d4cf-qr8r6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-c8537c-8665fb9775-qkjd9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-501d63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning ProbeError pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w4-f01de5-openshift-default-75bf69d85f-v6jb6 Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-501d63-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-f01de5-6cc545fd87-zxq6z Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Stopping container payload-pre-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f4a265-6c7585d45f-zmhsc Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-ipp-b-f4a265-5db88cf775-7jnq2 Stopping container payload-processing 20m Normal Pulled pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-processing-e2e-worker-w4-9f4dc9-76d5dd8489-6bczx Stopping container payload-processing 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f4a265-dc856b996 Created pod: payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f4a265 Scaled up replica set payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff from 0 to 1 20m Normal Started pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Container started 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Killing pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Stopping container payload-pre-processing 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d Add eth0 [10.134.1.36/23] from ovn-kubernetes 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck Add eth0 [10.134.1.37/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff Created pod: payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d 20m Normal Created pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Container created 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f4a265 Scaled up replica set payload-processing-e2e-ipp-b-f4a265-dc856b996 from 0 to 1 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff Error creating: pods "payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f4a265: serviceaccount "payload-processing-e2e-ipp-b-f4a265" not found 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f4a265-dc856b996 Error creating: pods "payload-processing-e2e-ipp-b-f4a265-dc856b996-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f4a265: serviceaccount "payload-processing-e2e-ipp-b-f4a265" not found 20m Normal Pulled pod/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d Container created 20m Normal Started pod/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck Container started 20m Normal Created pod/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck Container created 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-f01de5-65884f55d5-pbgx8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d Container started 19m Warning ProbeError pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Startup probe error: Get "http://10.134.1.34:15021/healthz/ready": dial tcp 10.134.1.34:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Startup probe failed: Get "http://10.134.1.34:15021/healthz/ready": dial tcp 10.134.1.34:15021: connect: connection refused 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f4a265-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f4a265-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-dfff59-749f5d5d6c-tc5d4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-worker-w1-dfff59-688dc9df89-4vk2r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-9f4dc9-7d9b75d459-qhb7s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Stopping container istio-proxy 19m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning ProbeError pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-5gq9q Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w1-dfff59-openshift-default-5fd958c56c-nfl22 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-8qf9k Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/e2e-worker-w4-9f4dc9-openshift-default-66cbb8f7f5-9v4pv Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Readiness probe failed: dial tcp 10.134.1.33:9004: connect: connection refused 19m Normal Killing pod/payload-processing-e2e-ipp-a-501d63-5fd9cc8bfb-krlt9 Stopping container payload-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Readiness probe failed: dial tcp 10.134.1.32:9004: connect: connection refused 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-501d63-678bbff655-z866s Stopping container payload-pre-processing 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f4a265-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f4a265-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Normal Killing pod/e2e-ipp-a-501d63-openshift-default-b85b799f7-hjm4b Stopping container istio-proxy 19m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-b-f4a265 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-f4a265: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-f4a265": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-f4a265, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b316f90e-4600-47b1-8608-52445d24a1a4, UID in object meta: 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f4a265-7c6884bdff-d444d Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-f4a265-dc856b996-7fdck Stopping container payload-processing 19m Normal Killing pod/e2e-ipp-b-f4a265-openshift-default-544bb4948f-mzs9d Stopping container istio-proxy 18m Normal Killing pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Stopping container payload-pre-processing 18m Normal Killing pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-9mgr7 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-gn6sw Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-nv6w4 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w5-383fde-openshift-default-5bf8ffbd59-hxkzr Stopping container istio-proxy 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-383fde-6f8cc9c655-m67lx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Stopping container payload-pre-processing 18m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-383fde-8486d66857-b9mwl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Stopping container payload-processing 18m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-55b72 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-f4d8p Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-jfcdr Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-thzz2 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w5-a9a967-openshift-default-57f7644bd5-vdmvv Stopping container istio-proxy 18m Normal Killing pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Stopping container payload-pre-processing 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-a9a967-5dd77f664c-4v5m2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-a9a967-6948b69c67-tgrtj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-worker-w5-164db6-7848484d55-lzjsq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-kw58b Stopping container istio-proxy 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-164db6-649d7fd8f5-2ttc6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Stopping container istio-proxy 17m Warning ProbeError pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-worker-w5-164db6-openshift-default-5c5fb4d6c7-r2t4j Readiness probe failed: HTTP probe failed with statuscode: 503 17m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 17m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-27n7m 17m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 17m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-27n7m Stopping container discovery 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 17m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Container started 17m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Container created 17m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 3 to 4 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-65xj6 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Add eth0 [10.134.1.39/23] from ovn-kubernetes 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Startup probe failed: Get "http://10.134.1.39:15021/healthz/ready": dial tcp 10.134.1.39:15021: connect: connection refused 17m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Startup probe error: Get "http://10.134.1.39:15021/healthz/ready": dial tcp 10.134.1.39:15021: connect: connection refused... 17m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Startup probe error: Get "http://10.134.1.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Startup probe failed: Get "http://10.134.1.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 17m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Container started 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Add eth0 [10.132.0.195/23] from ovn-kubernetes 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 4 to 5 17m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Container created 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 17m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-vhjzm 17m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Startup probe error: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused... 17m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Startup probe failed: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Readiness probe error: Get "http://10.132.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-vhjzm Readiness probe failed: Get "http://10.132.0.195:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Container created 15m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Container started 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-hmxgm 15m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Add eth0 [10.133.1.76/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-s62zq 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-gnmrk 15m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-7f5667f86c from 5 to 10 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Add eth0 [10.133.1.75/23] from ovn-kubernetes 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-r9wbx 15m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Add eth0 [10.134.1.42/23] from ovn-kubernetes 15m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Container created 15m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-7f5667f86c Created pod: maas-default-gateway-openshift-default-7f5667f86c-lfl9n 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Add eth0 [10.134.1.41/23] from ovn-kubernetes 15m Normal Pulled pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 15m Normal AddedInterface pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Add eth0 [10.132.0.196/23] from ovn-kubernetes 15m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Container created 15m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Container started 15m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Container created 15m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Container started 15m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Container started 15m Normal Created pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Container created 15m Normal Started pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Container started 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Startup probe error: Get "http://10.133.1.76:15021/healthz/ready": dial tcp 10.133.1.76:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Startup probe failed: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Startup probe failed: Get "http://10.134.1.42:15021/healthz/ready": dial tcp 10.134.1.42:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-lfl9n Startup probe error: Get "http://10.134.1.42:15021/healthz/ready": dial tcp 10.134.1.42:15021: connect: connection refused... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Startup probe error: Get "http://10.134.1.41:15021/healthz/ready": dial tcp 10.134.1.41:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Startup probe failed: Get "http://10.134.1.41:15021/healthz/ready": dial tcp 10.134.1.41:15021: connect: connection refused 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Startup probe failed: Get "http://10.133.1.76:15021/healthz/ready": dial tcp 10.133.1.76:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-s62zq Startup probe error: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Startup probe failed: Get "http://10.133.1.75:15021/healthz/ready": dial tcp 10.133.1.75:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-gnmrk Startup probe error: Get "http://10.133.1.75:15021/healthz/ready": dial tcp 10.133.1.75:15021: connect: connection refused... 15m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 15m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-2qs8x 15m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Readiness probe failed: Get "http://10.132.0.110:8080/ready": dial tcp 10.132.0.110:8080: connect: connection refused 15m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Stopping container discovery 15m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-2qs8x Readiness probe error: Get "http://10.132.0.110:8080/ready": dial tcp 10.132.0.110:8080: connect: connection refused... 15m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 14m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Readiness probe failed: Get "http://10.134.1.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 14m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-65xj6 Readiness probe error: Get "http://10.134.1.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 14m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Readiness probe failed: Get "http://10.134.1.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 14m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-hmxgm Readiness probe error: Get "http://10.134.1.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Readiness probe error: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-9rb5l Readiness probe failed: Get "http://10.132.0.33:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Readiness probe failed: Get "http://10.134.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 11m Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-8s26k Readiness probe error: Get "http://10.134.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 11m Normal AddedLabel gateway/e2e-worker-single-main-ee2b3d Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-ee2b3d 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-ee2b3d-openshift-default Scaled up replica set e2e-worker-single-main-ee2b3d-openshift-default-75656657cc from 0 to 1 11m Normal AddedInterface pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Add eth0 [10.132.0.197/23] from ovn-kubernetes 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-ee2b3d-openshift-default-75656657cc Created pod: e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-ee2b3d Scaled up replica set payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7 from 0 to 1 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-ee2b3d Scaled up replica set payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67c96bc from 0 to 1 11m Normal Created pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Container created 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7 Created pod: payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67c96bc Created pod: payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Add eth0 [10.134.1.44/23] from ovn-kubernetes 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67c96bc Error creating: pods "payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67c96bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ee2b3d: serviceaccount "payload-processing-e2e-worker-single-main-ee2b3d" not found 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7 Error creating: pods "payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-ee2b3d: serviceaccount "payload-processing-e2e-worker-single-main-ee2b3d" not found 11m Normal Started pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Container started 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Add eth0 [10.134.1.43/23] from ovn-kubernetes 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Container started 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Container created 11m Normal Created pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Container created 11m Normal Started pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Container started 11m Normal Pulled pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Warning ProbeError pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 11m Warning Unhealthy pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-ee2b3d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-ee2b3d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Normal Killing pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Stopping container payload-processing 10m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Stopping container payload-pre-processing 10m Normal Killing pod/e2e-worker-single-main-ee2b3d-openshift-default-75656657ccjfrc5 Stopping container istio-proxy 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-ee2b3d-58c9fb6f7rkpzn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-ee2b3d-59d67jgrpx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m4s Normal ScalingReplicaSet deployment/e2e-worker-single-main-230d0a-openshift-default Scaled up replica set e2e-worker-single-main-230d0a-openshift-default-55df6cdc97 from 0 to 1 8m4s Normal AddedLabel gateway/e2e-worker-single-main-230d0a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-230d0a 8m4s Normal SuccessfulCreate replicaset/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97 Created pod: e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 8m3s Normal AddedInterface pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Add eth0 [10.132.0.201/23] from ovn-kubernetes 7m59s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-230d0a Scaled up replica set payload-processing-e2e-worker-single-main-230d0a-5b7bc975f9 from 0 to 1 7m59s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6c667 Error creating: pods "payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6c667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-230d0a: serviceaccount "payload-processing-e2e-worker-single-main-230d0a" not found 7m59s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6c667 Created pod: payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k 7m59s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Add eth0 [10.134.1.46/23] from ovn-kubernetes 7m59s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-230d0a-5b7bc975f9 Created pod: payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp 7m59s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-230d0a-5b7bc975f9 Error creating: pods "payload-processing-e2e-worker-single-main-230d0a-5b7bc975f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-230d0a: serviceaccount "payload-processing-e2e-worker-single-main-230d0a" not found 7m59s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-230d0a Scaled up replica set payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6c667 from 0 to 1 7m58s Normal Created pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Container created 7m58s Normal Started pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Container started 7m58s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m58s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Add eth0 [10.133.1.96/23] from ovn-kubernetes 7m58s Normal Pulled pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m58s Normal Created pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Container created 7m58s Normal Started pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Container started 7m47s Warning ProbeError pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Readiness probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 7m47s Warning Unhealthy pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Readiness probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 7m46s Normal Started pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Container started 7m46s Normal Created pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Container created 7m46s Normal Pulled pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m41s Warning ProbeError pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Startup probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 7m41s Warning Unhealthy pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Startup probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 7m4s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-230d0a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m4s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-230d0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m52s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Stopping container payload-pre-processing 6m52s Normal Killing pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Stopping container payload-processing 6m45s Normal Killing pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Stopping container istio-proxy 6m40s Warning Unhealthy pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Readiness probe failed: HTTP probe failed with statuscode: 503 6m40s Warning ProbeError pod/e2e-worker-single-main-230d0a-openshift-default-55df6cdc97tx4f8 Readiness probe error: HTTP probe failed with statuscode: 503... 6m37s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-230d0a-5b7bc975flwttp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m37s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-230d0a-f4cc6zl88k Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m14s Normal AddedLabel gateway/e2e-ait-f9b881a4 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-f9b881a4 5m14s Normal ScalingReplicaSet deployment/e2e-ait-f9b881a4-openshift-default Scaled up replica set e2e-ait-f9b881a4-openshift-default-56bcf548ff from 0 to 1 5m14s Normal AddedInterface pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Add eth0 [10.134.1.47/23] from ovn-kubernetes 5m14s Normal SuccessfulCreate replicaset/e2e-ait-f9b881a4-openshift-default-56bcf548ff Created pod: e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c 5m10s Normal Pulled pod/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m10s Normal Created pod/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k Container created 5m10s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-f9b881a4-54c985c9cc Created pod: payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f 5m10s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-f9b881a4 Scaled up replica set payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f from 0 to 1 5m10s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f Created pod: payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k 5m10s Warning FailedCreate replicaset/payload-processing-e2e-ait-f9b881a4-54c985c9cc Error creating: pods "payload-processing-e2e-ait-f9b881a4-54c985c9cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f9b881a4: serviceaccount "payload-processing-e2e-ait-f9b881a4" not found 5m10s Normal Created pod/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f Container created 5m10s Normal Pulled pod/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m10s Normal AddedInterface pod/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f Add eth0 [10.134.1.49/23] from ovn-kubernetes 5m10s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-f9b881a4 Scaled up replica set payload-processing-e2e-ait-f9b881a4-54c985c9cc from 0 to 1 5m10s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f Error creating: pods "payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-f9b881a4: serviceaccount "payload-processing-e2e-ait-f9b881a4" not found 5m10s Normal AddedInterface pod/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k Add eth0 [10.133.1.102/23] from ovn-kubernetes 5m10s Normal Started pod/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k Container started 5m9s Normal Started pod/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f Container started 4m59s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-f9b881a4-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m59s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-f9b881a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 4m58s Normal Created pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Container created 4m58s Normal Started pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Container started 4m58s Normal Pulled pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m52s Warning ProbeError pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Startup probe error: Get "http://10.134.1.47:15021/healthz/ready": dial tcp 10.134.1.47:15021: connect: connection refused... 4m52s Warning Unhealthy pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Startup probe failed: Get "http://10.134.1.47:15021/healthz/ready": dial tcp 10.134.1.47:15021: connect: connection refused 4m50s Normal Killing pod/payload-pre-processing-e2e-ait-f9b881a4-6f54c8697f-54m2k Stopping container payload-pre-processing 4m50s Normal Killing pod/payload-processing-e2e-ait-f9b881a4-54c985c9cc-w8w2f Stopping container payload-processing 4m44s Normal Killing pod/e2e-ait-f9b881a4-openshift-default-56bcf548ff-rcf7c Stopping container istio-proxy 4m43s Normal SuccessfulCreate replicaset/e2e-authcfg-http-c6e992-openshift-default-6868c659d9 Created pod: e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz 4m43s Normal ScalingReplicaSet deployment/e2e-authcfg-http-c6e992-openshift-default Scaled up replica set e2e-authcfg-http-c6e992-openshift-default-6868c659d9 from 0 to 1 4m43s Normal AddedLabel gateway/e2e-authcfg-http-c6e992 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-c6e992 4m42s Normal AddedInterface pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Add eth0 [10.134.1.51/23] from ovn-kubernetes 4m38s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f Error creating: pods "payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-c6e992: serviceaccount "payload-processing-e2e-authcfg-http-c6e992" not found 4m38s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f Created pod: payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk 4m38s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-c6e992 Scaled up replica set payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f from 0 to 1 4m38s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f Error creating: pods "payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-c6e992: serviceaccount "payload-processing-e2e-authcfg-http-c6e992" not found 4m38s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-c6e992 Scaled up replica set payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f from 0 to 1 4m38s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f Created pod: payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk 4m38s Normal Created pod/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk Container created 4m38s Normal Pulled pod/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m38s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk Add eth0 [10.134.1.54/23] from ovn-kubernetes 4m38s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk Add eth0 [10.134.1.53/23] from ovn-kubernetes 4m38s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m37s Normal Started pod/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk Container started 4m37s Normal Created pod/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk Container created 4m37s Normal Started pod/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk Container started 4m26s Warning Unhealthy pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Readiness probe failed: Get "http://10.134.1.51:15021/healthz/ready": dial tcp 10.134.1.51:15021: connect: connection refused 4m26s Warning ProbeError pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Readiness probe error: Get "http://10.134.1.51:15021/healthz/ready": dial tcp 10.134.1.51:15021: connect: connection refused... 4m26s Normal Pulled pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m25s Normal Started pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Container started 4m25s Normal Created pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Container created 4m20s Warning Unhealthy pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Startup probe failed: Get "http://10.134.1.51:15021/healthz/ready": dial tcp 10.134.1.51:15021: connect: connection refused 4m20s Warning ProbeError pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Startup probe error: Get "http://10.134.1.51:15021/healthz/ready": dial tcp 10.134.1.51:15021: connect: connection refused... 4m13s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-c6e992-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m13s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-c6e992-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m58s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-c6e992-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 3m58s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-c6e992-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 2m55s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-c6e992-6d6f87545f-k7nxk Stopping container payload-pre-processing 2m55s Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-authcfg-http-c6e992 Failed to update endpoint openshift-ingress/payload-processing-e2e-authcfg-http-c6e992: Operation cannot be fulfilled on endpoints "payload-processing-e2e-authcfg-http-c6e992": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-authcfg-http-c6e992, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 7bbb4351-330b-4be2-8b40-382633fd42c9, UID in object meta: 2m55s Normal Killing pod/payload-processing-e2e-authcfg-http-c6e992-55b58d7f9f-zzkqk Stopping container payload-processing 2m42s Normal Killing pod/e2e-authcfg-http-c6e992-openshift-default-6868c659d9-ncmwz Stopping container istio-proxy 2m32s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Readiness probe failed: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m32s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-v67pm Readiness probe error: Get "http://10.133.0.39:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m15s Warning Unhealthy pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Readiness probe failed: Get "http://10.133.1.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m15s Warning ProbeError pod/maas-default-gateway-openshift-default-7f5667f86c-r9wbx Readiness probe error: Get "http://10.133.1.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...