=== events in openshift-ingress (2026-11-01T22:50:48Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c to ip-10-0-135-146.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg Successfully assigned openshift-ingress/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 33m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h to ip-10-0-135-146.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p to ip-10-0-135-146.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Successfully assigned openshift-ingress/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm to ip-10-0-131-110.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 to ip-10-0-136-47.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Successfully assigned openshift-ingress/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 to ip-10-0-135-146.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts to ip-10-0-131-110.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Successfully assigned openshift-ingress/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf to ip-10-0-135-146.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Successfully assigned openshift-ingress/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 to ip-10-0-131-110.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p to ip-10-0-131-110.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 30m Normal Scheduled pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Successfully assigned openshift-ingress/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j to ip-10-0-136-47.us-east-2.compute.internal 41m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq to ip-10-0-135-146.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s to ip-10-0-135-146.us-east-2.compute.internal 33m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm to ip-10-0-136-47.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 36m Normal Scheduled pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Successfully assigned openshift-ingress/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 51m Normal Scheduled pod/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 Successfully assigned openshift-ingress/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 to ip-10-0-131-110.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Successfully assigned openshift-ingress/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx to ip-10-0-131-110.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv to ip-10-0-136-47.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw to ip-10-0-135-146.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 to ip-10-0-135-146.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 to ip-10-0-131-110.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww to ip-10-0-136-47.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Successfully assigned openshift-ingress/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw to ip-10-0-135-146.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n to ip-10-0-135-146.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-ait-93a347eb-openshift-default-95778769c-7g26j Successfully assigned openshift-ingress/e2e-ait-93a347eb-openshift-default-95778769c-7g26j to ip-10-0-136-47.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Successfully assigned openshift-ingress/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch to ip-10-0-131-110.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn to ip-10-0-136-47.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f to ip-10-0-135-146.us-east-2.compute.internal 41m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 50m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt to ip-10-0-136-47.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f to ip-10-0-131-110.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr to ip-10-0-135-146.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Successfully assigned openshift-ingress/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd to ip-10-0-136-47.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg to ip-10-0-131-110.us-east-2.compute.internal 5m11s Normal Scheduled pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Successfully assigned openshift-ingress/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 36m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 to ip-10-0-136-47.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 to ip-10-0-136-47.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm to ip-10-0-136-47.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Successfully assigned openshift-ingress/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz to ip-10-0-136-47.us-east-2.compute.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm to ip-10-0-135-146.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg to ip-10-0-135-146.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-681db477-64946f668d-mf6dm Successfully assigned openshift-ingress/payload-processing-e2e-ait-681db477-64946f668d-mf6dm to ip-10-0-136-47.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc to ip-10-0-135-146.us-east-2.compute.internal 66m Normal Scheduled pod/router-default-554f8bcb97-6mq5w Successfully assigned openshift-ingress/router-default-554f8bcb97-6mq5w to ip-10-0-131-110.us-east-2.compute.internal 67m Warning FailedScheduling pod/router-default-554f8bcb97-6mq5w no nodes available to schedule pods 43m Normal Scheduled pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Successfully assigned openshift-ingress/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv to ip-10-0-131-110.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx Successfully assigned openshift-ingress/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx to ip-10-0-135-146.us-east-2.compute.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh to ip-10-0-136-47.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll to ip-10-0-136-47.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Successfully assigned openshift-ingress/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 to ip-10-0-131-110.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Successfully assigned openshift-ingress/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 to ip-10-0-131-110.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt to ip-10-0-136-47.us-east-2.compute.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-8xkxn to ip-10-0-136-47.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Successfully assigned openshift-ingress/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw to ip-10-0-136-47.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m to ip-10-0-136-47.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 to ip-10-0-135-146.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr to ip-10-0-135-146.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l to ip-10-0-135-146.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Successfully assigned openshift-ingress/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 to ip-10-0-131-110.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-7bg9d to ip-10-0-135-146.us-east-2.compute.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 to ip-10-0-135-146.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Successfully assigned openshift-ingress/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 to ip-10-0-136-47.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Successfully assigned openshift-ingress/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f to ip-10-0-131-110.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm to ip-10-0-135-146.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 Successfully assigned openshift-ingress/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 to ip-10-0-136-47.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Successfully assigned openshift-ingress/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-5wxxh to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 to ip-10-0-135-146.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd Successfully assigned openshift-ingress/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd to ip-10-0-136-47.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n to ip-10-0-136-47.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj Successfully assigned openshift-ingress/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj to ip-10-0-135-146.us-east-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Successfully assigned openshift-ingress/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr to ip-10-0-136-47.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Successfully assigned openshift-ingress/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz to ip-10-0-131-110.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd to ip-10-0-136-47.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq to ip-10-0-136-47.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 to ip-10-0-135-146.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 Successfully assigned openshift-ingress/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 to ip-10-0-131-110.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t to ip-10-0-135-146.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Successfully assigned openshift-ingress/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb to ip-10-0-131-110.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Successfully assigned openshift-ingress/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 to ip-10-0-136-47.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww to ip-10-0-136-47.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 61m Normal Scheduled pod/kube-auth-proxy-b69dc6f4d-kdssh Successfully assigned openshift-ingress/kube-auth-proxy-b69dc6f4d-kdssh to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq to ip-10-0-136-47.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 to ip-10-0-136-47.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 to ip-10-0-131-110.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Successfully assigned openshift-ingress/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls to ip-10-0-131-110.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 54m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm to ip-10-0-131-110.us-east-2.compute.internal 61m Normal Scheduled pod/kube-auth-proxy-b69dc6f4d-9rkd4 Successfully assigned openshift-ingress/kube-auth-proxy-b69dc6f4d-9rkd4 to ip-10-0-136-47.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw Successfully assigned openshift-ingress/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw to ip-10-0-136-47.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx to ip-10-0-136-47.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Successfully assigned openshift-ingress/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s to ip-10-0-136-47.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Successfully assigned openshift-ingress/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 to ip-10-0-136-47.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 to ip-10-0-135-146.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm Successfully assigned openshift-ingress/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm to ip-10-0-136-47.us-east-2.compute.internal 62m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw to ip-10-0-131-110.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb to ip-10-0-136-47.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf to ip-10-0-131-110.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Successfully assigned openshift-ingress/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k to ip-10-0-131-110.us-east-2.compute.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 to ip-10-0-136-47.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg Successfully assigned openshift-ingress/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg to ip-10-0-131-110.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv to ip-10-0-131-110.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Successfully assigned openshift-ingress/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 to ip-10-0-131-110.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 to ip-10-0-131-110.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 to ip-10-0-136-47.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 to ip-10-0-131-110.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 to ip-10-0-135-146.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Successfully assigned openshift-ingress/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 to ip-10-0-131-110.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh to ip-10-0-136-47.us-east-2.compute.internal 60m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-v64b4 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-v64b4 to ip-10-0-136-47.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j to ip-10-0-135-146.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Successfully assigned openshift-ingress/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm to ip-10-0-131-110.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k to ip-10-0-131-110.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj to ip-10-0-135-146.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-7b98bb678c-dxpk7 Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-dxpk7 to ip-10-0-136-47.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 to ip-10-0-135-146.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz to ip-10-0-135-146.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 to ip-10-0-131-110.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 to ip-10-0-136-47.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 to ip-10-0-135-146.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Successfully assigned openshift-ingress/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz to ip-10-0-131-110.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-dsdmp to ip-10-0-135-146.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-b-004180-548d486474-4m87j Successfully assigned openshift-ingress/payload-processing-e2e-api-b-004180-548d486474-4m87j to ip-10-0-135-146.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf to ip-10-0-136-47.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb to ip-10-0-131-110.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt to ip-10-0-131-110.us-east-2.compute.internal 23m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 to ip-10-0-136-47.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 to ip-10-0-131-110.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 to ip-10-0-136-47.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-gj2hp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-gj2hp 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-m6nc9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-m6nc9 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Successfully assigned openshift-ingress/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc to ip-10-0-131-110.us-east-2.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-8dg8l to ip-10-0-131-110.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd to ip-10-0-135-146.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh to ip-10-0-135-146.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 to ip-10-0-136-47.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr to ip-10-0-131-110.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw to ip-10-0-136-47.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm to ip-10-0-131-110.us-east-2.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-782nd to ip-10-0-136-47.us-east-2.compute.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 to ip-10-0-135-146.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-twqtc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-twqtc 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 22m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf to ip-10-0-131-110.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Normal Scheduled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Successfully assigned openshift-ingress/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Successfully assigned openshift-ingress/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz to ip-10-0-131-110.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 to ip-10-0-136-47.us-east-2.compute.internal 32m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x to ip-10-0-135-146.us-east-2.compute.internal 62m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-zpvwt Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-zpvwt to ip-10-0-136-47.us-east-2.compute.internal 5m7s Normal Scheduled pod/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm Successfully assigned openshift-ingress/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm to ip-10-0-136-47.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w to ip-10-0-131-110.us-east-2.compute.internal 4m35s Normal Scheduled pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Successfully assigned openshift-ingress/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n to ip-10-0-135-146.us-east-2.compute.internal 73m Warning FailedScheduling pod/router-default-554f8bcb97-6mq5w no nodes available to schedule pods 61m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6455b4886ft4x6p Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6455b4886ft4x6p to ip-10-0-131-110.us-east-2.compute.internal 39m Warning FailedScheduling pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 36m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 48m Normal Scheduled pod/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x to ip-10-0-135-146.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 to ip-10-0-135-146.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Successfully assigned openshift-ingress/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm to ip-10-0-131-110.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr to ip-10-0-136-47.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n to ip-10-0-136-47.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d to ip-10-0-131-110.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Successfully assigned openshift-ingress/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Successfully assigned openshift-ingress/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d to ip-10-0-131-110.us-east-2.compute.internal 37m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 50m Normal Scheduled pod/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh Successfully assigned openshift-ingress/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh to ip-10-0-135-146.us-east-2.compute.internal 5m7s Normal Scheduled pod/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv to ip-10-0-136-47.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Successfully assigned openshift-ingress/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp to ip-10-0-135-146.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l to ip-10-0-131-110.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw to ip-10-0-136-47.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv to ip-10-0-135-146.us-east-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 to ip-10-0-135-146.us-east-2.compute.internal 8m Normal Scheduled pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Successfully assigned openshift-ingress/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 to ip-10-0-136-47.us-east-2.compute.internal 36m Normal Scheduled pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx to ip-10-0-136-47.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr Successfully assigned openshift-ingress/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 to ip-10-0-131-110.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk to ip-10-0-131-110.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Successfully assigned openshift-ingress/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Successfully assigned openshift-ingress/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj to ip-10-0-135-146.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz to ip-10-0-136-47.us-east-2.compute.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 to ip-10-0-136-47.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Successfully assigned openshift-ingress/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd to ip-10-0-131-110.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Successfully assigned openshift-ingress/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp to ip-10-0-136-47.us-east-2.compute.internal 11m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv to ip-10-0-136-47.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Successfully assigned openshift-ingress/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Successfully assigned openshift-ingress/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl to ip-10-0-135-146.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz to ip-10-0-135-146.us-east-2.compute.internal 11m Normal Scheduled pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Successfully assigned openshift-ingress/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz to ip-10-0-136-47.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr to ip-10-0-135-146.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Successfully assigned openshift-ingress/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn to ip-10-0-131-110.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Successfully assigned openshift-ingress/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv to ip-10-0-131-110.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-m9bdz to ip-10-0-131-110.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs to ip-10-0-131-110.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw to ip-10-0-136-47.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl to ip-10-0-136-47.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Successfully assigned openshift-ingress/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l to ip-10-0-135-146.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Successfully assigned openshift-ingress/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 to ip-10-0-131-110.us-east-2.compute.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 to ip-10-0-136-47.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t to ip-10-0-136-47.us-east-2.compute.internal 11m Normal Scheduled pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 to ip-10-0-135-146.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt to ip-10-0-135-146.us-east-2.compute.internal 7m56s Normal Scheduled pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 to ip-10-0-135-146.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g to ip-10-0-136-47.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx to ip-10-0-135-146.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Successfully assigned openshift-ingress/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g to ip-10-0-135-146.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Successfully assigned openshift-ingress/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-mrlcb to ip-10-0-135-146.us-east-2.compute.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng to ip-10-0-136-47.us-east-2.compute.internal 40m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz to ip-10-0-131-110.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Successfully assigned openshift-ingress/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c to ip-10-0-136-47.us-east-2.compute.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Successfully assigned openshift-ingress/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq to ip-10-0-131-110.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl to ip-10-0-131-110.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw to ip-10-0-136-47.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 to ip-10-0-136-47.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk to ip-10-0-135-146.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp to ip-10-0-135-146.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g to ip-10-0-136-47.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 to ip-10-0-135-146.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-derive-d5713372-5555895f6-lk42f Successfully assigned openshift-ingress/payload-processing-e2e-derive-d5713372-5555895f6-lk42f to ip-10-0-136-47.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx to ip-10-0-135-146.us-east-2.compute.internal 40m Normal Scheduled pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Successfully assigned openshift-ingress/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 to ip-10-0-131-110.us-east-2.compute.internal 40m Warning FailedScheduling pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 24m Normal Scheduled pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Successfully assigned openshift-ingress/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n to ip-10-0-135-146.us-east-2.compute.internal 41m Warning FailedScheduling pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 32m Normal Scheduled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Successfully assigned openshift-ingress/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv to ip-10-0-135-146.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Successfully assigned openshift-ingress/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-vktwt to ip-10-0-136-47.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg to ip-10-0-136-47.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 to ip-10-0-135-146.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 to ip-10-0-136-47.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Successfully assigned openshift-ingress/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq to ip-10-0-131-110.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 to ip-10-0-135-146.us-east-2.compute.internal 37m Warning FailedScheduling pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h to ip-10-0-131-110.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 Successfully assigned openshift-ingress/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 to ip-10-0-136-47.us-east-2.compute.internal 37m Warning FailedScheduling pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 47m Normal Scheduled pod/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h to ip-10-0-135-146.us-east-2.compute.internal 36m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts to ip-10-0-136-47.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb to ip-10-0-135-146.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc to ip-10-0-135-146.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq to ip-10-0-136-47.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd to ip-10-0-135-146.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Successfully assigned openshift-ingress/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj to ip-10-0-135-146.us-east-2.compute.internal 7m56s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp to ip-10-0-136-47.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Successfully assigned openshift-ingress/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 to ip-10-0-135-146.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld to ip-10-0-131-110.us-east-2.compute.internal 57m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-c8vgk to ip-10-0-136-47.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Successfully assigned openshift-ingress/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g to ip-10-0-131-110.us-east-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 to ip-10-0-135-146.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Successfully assigned openshift-ingress/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt to ip-10-0-131-110.us-east-2.compute.internal 62m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-p4lpt to ip-10-0-131-110.us-east-2.compute.internal 4m30s Normal Scheduled pod/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Successfully assigned openshift-ingress/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt to ip-10-0-131-110.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w to ip-10-0-131-110.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Successfully assigned openshift-ingress/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng to ip-10-0-131-110.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-tmn2k to ip-10-0-131-110.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw to ip-10-0-131-110.us-east-2.compute.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn to ip-10-0-136-47.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 to ip-10-0-136-47.us-east-2.compute.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph to ip-10-0-131-110.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Successfully assigned openshift-ingress/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq to ip-10-0-135-146.us-east-2.compute.internal 32m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c to ip-10-0-135-146.us-east-2.compute.internal 42m Normal Scheduled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Successfully assigned openshift-ingress/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 to ip-10-0-131-110.us-east-2.compute.internal 32m Normal Scheduled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Successfully assigned openshift-ingress/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz to ip-10-0-136-47.us-east-2.compute.internal 40m Normal Scheduled pod/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf to ip-10-0-131-110.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 to ip-10-0-136-47.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Successfully assigned openshift-ingress/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv to ip-10-0-135-146.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n to ip-10-0-136-47.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Successfully assigned openshift-ingress/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c to ip-10-0-131-110.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 32m Normal Scheduled pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Successfully assigned openshift-ingress/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq to ip-10-0-131-110.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Successfully assigned openshift-ingress/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw to ip-10-0-131-110.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz to ip-10-0-135-146.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-cjw9g to ip-10-0-136-47.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Successfully assigned openshift-ingress/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw to ip-10-0-136-47.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq to ip-10-0-135-146.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl to ip-10-0-135-146.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Successfully assigned openshift-ingress/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p to ip-10-0-131-110.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln to ip-10-0-135-146.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Successfully assigned openshift-ingress/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg to ip-10-0-131-110.us-east-2.compute.internal 42m Normal Scheduled pod/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc Successfully assigned openshift-ingress/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc to ip-10-0-136-47.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m to ip-10-0-135-146.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b to ip-10-0-136-47.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh to ip-10-0-135-146.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv to ip-10-0-131-110.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-7bb9489676-tfddh Successfully assigned openshift-ingress/payload-processing-7bb9489676-tfddh to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Successfully assigned openshift-ingress/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd to ip-10-0-135-146.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk to ip-10-0-131-110.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf to ip-10-0-136-47.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp to ip-10-0-131-110.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv to ip-10-0-136-47.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf to ip-10-0-135-146.us-east-2.compute.internal 42m Normal Scheduled pod/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv to ip-10-0-136-47.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq to ip-10-0-131-110.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Successfully assigned openshift-ingress/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 to ip-10-0-131-110.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht to ip-10-0-135-146.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 to ip-10-0-135-146.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv to ip-10-0-131-110.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c to ip-10-0-136-47.us-east-2.compute.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-shr8g to ip-10-0-131-110.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Successfully assigned openshift-ingress/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 to ip-10-0-136-47.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn to ip-10-0-136-47.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Successfully assigned openshift-ingress/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv to ip-10-0-131-110.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp to ip-10-0-131-110.us-east-2.compute.internal 19m Normal Scheduled pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Successfully assigned openshift-ingress/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr to ip-10-0-136-47.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 to ip-10-0-131-110.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng to ip-10-0-131-110.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz to ip-10-0-135-146.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 to ip-10-0-131-110.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Successfully assigned openshift-ingress/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm to ip-10-0-135-146.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv Successfully assigned openshift-ingress/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv to ip-10-0-136-47.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5bc884dd87-prd6f to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Successfully assigned openshift-ingress/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb to ip-10-0-136-47.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj to ip-10-0-135-146.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 to ip-10-0-136-47.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 to ip-10-0-136-47.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg to ip-10-0-136-47.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Successfully assigned openshift-ingress/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs to ip-10-0-131-110.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k to ip-10-0-136-47.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p to ip-10-0-135-146.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 to ip-10-0-131-110.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m to ip-10-0-135-146.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 to ip-10-0-136-47.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Successfully assigned openshift-ingress/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z to ip-10-0-135-146.us-east-2.compute.internal 36m Warning FailedScheduling pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 32m Normal Scheduled pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Successfully assigned openshift-ingress/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb to ip-10-0-135-146.us-east-2.compute.internal 4m30s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf to ip-10-0-136-47.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Successfully assigned openshift-ingress/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 to ip-10-0-136-47.us-east-2.compute.internal 39m Warning FailedScheduling pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 28m Normal Scheduled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Successfully assigned openshift-ingress/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj to ip-10-0-131-110.us-east-2.compute.internal 56m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 to ip-10-0-136-47.us-east-2.compute.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 to ip-10-0-135-146.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Successfully assigned openshift-ingress/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw to ip-10-0-131-110.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Successfully assigned openshift-ingress/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb to ip-10-0-136-47.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Successfully assigned openshift-ingress/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz to ip-10-0-131-110.us-east-2.compute.internal 22m Normal Scheduled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Successfully assigned openshift-ingress/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p to ip-10-0-131-110.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 to ip-10-0-136-47.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr to ip-10-0-135-146.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl to ip-10-0-135-146.us-east-2.compute.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 to ip-10-0-135-146.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh to ip-10-0-135-146.us-east-2.compute.internal 56m Normal Scheduled pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g to ip-10-0-136-47.us-east-2.compute.internal 73m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 73m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 73m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-554f8bcb97 from 0 to 1 73m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 73m Normal SuccessfulCreate replicaset/router-default-554f8bcb97 Created pod: router-default-554f8bcb97-6mq5w 67m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 66m Warning FailedMount pod/router-default-554f8bcb97-6mq5w MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Warning FailedMount pod/router-default-554f8bcb97-6mq5w MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Warning FailedMount pod/router-default-554f8bcb97-6mq5w MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 66m Warning FailedMount pod/router-default-554f8bcb97-6mq5w MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 66m Normal Created pod/router-default-554f8bcb97-6mq5w Container created 66m Normal Started pod/router-default-554f8bcb97-6mq5w Container started 66m Normal Pulled pod/router-default-554f8bcb97-6mq5w Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 66m Normal AddedInterface pod/router-default-554f8bcb97-6mq5w Add eth0 [10.132.0.10/23] from ovn-kubernetes 62m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 62m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-zpvwt Add eth0 [10.133.0.21/23] from ovn-kubernetes 62m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 62m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-zpvwt Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 62m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-zpvwt 62m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-zpvwt Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.663s (3.663s including waiting). Image size: 185476477 bytes. 62m Normal Started pod/istiod-openshift-gateway-7876bfcb74-zpvwt Container started 62m Normal Created pod/istiod-openshift-gateway-7876bfcb74-zpvwt Container created 62m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 0 to 1 62m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-p4lpt 62m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Add eth0 [10.132.0.35/23] from ovn-kubernetes 62m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 62m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 62m Warning ProbeError pod/istiod-openshift-gateway-7876bfcb74-zpvwt Readiness probe error: HTTP probe failed with statuscode: 503... 62m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-64695b4bc9 Created pod: openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw 62m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw Add eth0 [10.132.0.36/23] from ovn-kubernetes 62m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-64695b4bc9 from 0 to 1 62m Warning Unhealthy pod/istiod-openshift-gateway-7876bfcb74-zpvwt Readiness probe failed: HTTP probe failed with statuscode: 503 62m Normal Pulling pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 62m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 62m Normal Started pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw Container started 62m Normal Created pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw Container created 62m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-64695b4bc9-gzknw Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.138s (8.138s including waiting). Image size: 183555333 bytes. 62m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 8.022s (8.023s including waiting). Image size: 183555333 bytes. 62m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 62m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Normal SuccessfulCreate replicaset/kube-auth-proxy-b69dc6f4d Created pod: kube-auth-proxy-b69dc6f4d-9rkd4 61m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6455b4886ft4x6p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 61m Normal SuccessfulCreate replicaset/kube-auth-proxy-b69dc6f4d Created pod: kube-auth-proxy-b69dc6f4d-kdssh 61m Warning FailedMount pod/kube-auth-proxy-b69dc6f4d-9rkd4 MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found 61m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6455b4886f Created pod: data-science-gateway-data-science-gateway-class-6455b4886ft4x6p 61m Warning FailedMount pod/kube-auth-proxy-b69dc6f4d-kdssh MountVolume.SetUp failed for volume "tls-certs" : secret "kube-auth-proxy-tls" not found 61m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6455b4886ft4x6p Add eth0 [10.132.0.43/23] from ovn-kubernetes 61m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6455b4886f from 0 to 1 61m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-b69dc6f4d from 0 to 2 61m Normal Created pod/data-science-gateway-data-science-gateway-class-6455b4886ft4x6p Container created 61m Normal Started pod/data-science-gateway-data-science-gateway-class-6455b4886ft4x6p Container started 61m Normal AddedInterface pod/kube-auth-proxy-b69dc6f4d-9rkd4 Add eth0 [10.133.0.23/23] from ovn-kubernetes 61m Normal Pulling pod/kube-auth-proxy-b69dc6f4d-kdssh Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 61m Normal AddedInterface pod/kube-auth-proxy-b69dc6f4d-kdssh Add eth0 [10.134.0.24/23] from ovn-kubernetes 61m Normal Pulling pod/kube-auth-proxy-b69dc6f4d-9rkd4 Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 61m Normal Created pod/kube-auth-proxy-b69dc6f4d-9rkd4 Container created 61m Normal Started pod/kube-auth-proxy-b69dc6f4d-9rkd4 Container started 61m Normal Pulled pod/kube-auth-proxy-b69dc6f4d-9rkd4 Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.505s (2.505s including waiting). Image size: 188904677 bytes. 61m Normal Pulled pod/kube-auth-proxy-b69dc6f4d-kdssh Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 3.065s (3.065s including waiting). Image size: 188904677 bytes. 61m Normal Created pod/kube-auth-proxy-b69dc6f4d-kdssh Container created 61m Normal Started pod/kube-auth-proxy-b69dc6f4d-kdssh Container started 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 61m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 60m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-v64b4 Add eth0 [10.133.0.30/23] from ovn-kubernetes 60m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-v64b4 60m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-v64b4 Container started 60m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-v64b4 Container created 60m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-v64b4 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 60m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 60m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-zpvwt 60m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-zpvwt Stopping container discovery 60m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 60m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Container created 60m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Container started 60m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Startup probe failed: Get "http://10.132.0.35:15021/healthz/ready": dial tcp 10.132.0.35:15021: connect: connection refused 60m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Startup probe error: Get "http://10.132.0.35:15021/healthz/ready": dial tcp 10.132.0.35:15021: connect: connection refused... 60m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 60m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 59m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-7bb9489676 from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-7bb9489676 Error creating: pods "payload-processing-7bb9489676-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 57m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-dxpk7 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 1 to 4 57m Normal Pulling pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 57m Normal SuccessfulCreate replicaset/payload-processing-7bb9489676 Created pod: payload-processing-7bb9489676-tfddh 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-vktwt 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Add eth0 [10.133.0.38/23] from ovn-kubernetes 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Add eth0 [10.133.0.37/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-vwrt5 57m Normal Pulling pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 57m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-mrlcb 57m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Add eth0 [10.134.0.35/23] from ovn-kubernetes 57m Normal Pulling pod/payload-processing-7bb9489676-tfddh Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal AddedInterface pod/payload-processing-7bb9489676-tfddh Add eth0 [10.133.0.40/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-dxpk7 Add eth0 [10.133.0.39/23] from ovn-kubernetes 57m Normal Pulling pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 57m Normal Pulling pod/payload-pre-processing-7b98bb678c-dxpk7 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal Pulled pod/payload-processing-7bb9489676-tfddh Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.076s (3.076s including waiting). Image size: 125353963 bytes. 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Container started 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Container created 57m Normal Created pod/payload-processing-7bb9489676-tfddh Container created 57m Normal Created pod/payload-pre-processing-7b98bb678c-dxpk7 Container created 57m Normal Pulled pod/payload-pre-processing-7b98bb678c-dxpk7 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.971s (2.971s including waiting). Image size: 125353963 bytes. 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.865s (3.865s including waiting). Image size: 183555333 bytes. 57m Normal Started pod/payload-pre-processing-7b98bb678c-dxpk7 Container started 57m Normal Started pod/payload-processing-7bb9489676-tfddh Container started 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Container started 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Container created 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.245s (4.245s including waiting). Image size: 183555333 bytes. 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Container started 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Container created 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 4.141s (4.141s including waiting). Image size: 183555333 bytes. 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Startup probe error: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Startup probe failed: Get "http://10.134.0.35:15021/healthz/ready": dial tcp 10.134.0.35:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Startup probe error: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Startup probe failed: Get "http://10.133.0.37:15021/healthz/ready": dial tcp 10.133.0.37:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Startup probe error: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Startup probe failed: Get "http://10.133.0.38:15021/healthz/ready": dial tcp 10.133.0.38:15021: connect: connection refused 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-hcdq8 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Add eth0 [10.132.0.53/23] from ovn-kubernetes 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Add eth0 [10.134.0.36/23] from ovn-kubernetes 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Add eth0 [10.133.0.42/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-m9bdz 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-7bg9d 57m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Add eth0 [10.134.0.37/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-5wxxh 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 4 to 8 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Container started 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Container created 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Container started 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Container created 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Container created 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Container started 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Container started 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Container created 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Startup probe error: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Startup probe failed: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Startup probe error: Get "http://10.132.0.53:15021/healthz/ready": dial tcp 10.132.0.53:15021: connect: connection refused... 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Startup probe error: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Startup probe failed: Get "http://10.134.0.36:15021/healthz/ready": dial tcp 10.134.0.36:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Startup probe error: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Startup probe failed: Get "http://10.133.0.42:15021/healthz/ready": dial tcp 10.133.0.42:15021: connect: connection refused 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Startup probe failed: Get "http://10.134.0.37:15021/healthz/ready": dial tcp 10.134.0.37:15021: connect: connection refused 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Add eth0 [10.132.0.54/23] from ovn-kubernetes 57m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-9sjh9 57m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 (combined from similar events): Created pod: maas-default-gateway-openshift-default-5bc884dd87-c8vgk 57m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 8 to 10 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Container created 57m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Add eth0 [10.133.0.43/23] from ovn-kubernetes 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Container created 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Container started 57m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Container started 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Startup probe failed: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Startup probe error: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused... 57m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Startup probe error: Get "http://10.132.0.54:15021/healthz/ready": dial tcp 10.132.0.54:15021: connect: connection refused... 57m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Startup probe failed: Get "http://10.133.0.43:15021/healthz/ready": dial tcp 10.133.0.43:15021: connect: connection refused 57m Normal AddedLabel gateway/e2e-worker-w3-ff3e70 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-ff3e70 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled up replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 0 to 1 57m Normal AddedLabel gateway/e2e-worker-w5-b0b7a3 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-b0b7a3 57m Normal AddedLabel gateway/e2e-worker-w1-53b784 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-53b784 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled up replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-f033b1-openshift-default Scaled up replica set e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 from 0 to 1 57m Normal AddedLabel gateway/e2e-worker-w2-f033b1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-f033b1 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled up replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 Created pod: e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p 57m Normal AddedLabel gateway/e2e-worker-w4-0def36 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-0def36 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm 57m Normal AddedLabel gateway/e2e-worker-w6-fb9597 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-fb9597 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z 57m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled up replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 0 to 1 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled up replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled up replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 0 to 1 57m Normal AddedLabel gateway/e2e-worker-w0-1f3a6b Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-1f3a6b 57m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Add eth0 [10.133.0.44/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Add eth0 [10.134.0.41/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Add eth0 [10.134.0.39/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Add eth0 [10.133.0.45/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Add eth0 [10.134.0.38/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Add eth0 [10.134.0.40/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Add eth0 [10.133.0.46/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-fb9597-86489d9d58 Error creating: pods "payload-processing-e2e-worker-w6-fb9597-86489d9d58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-fb9597: serviceaccount "payload-processing-e2e-worker-w6-fb9597" not found 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-1f3a6b-686494f97b Error creating: pods "payload-processing-e2e-worker-w0-1f3a6b-686494f97b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-1f3a6b: serviceaccount "payload-processing-e2e-worker-w0-1f3a6b" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c Created pod: payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-fb9597-86489d9d58 Created pod: payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n 56m Normal Created pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Container created 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c Error creating: pods "payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-1f3a6b: serviceaccount "payload-processing-e2e-worker-w0-1f3a6b" not found 56m Normal Pulled pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-1f3a6b Scaled up replica set payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c from 0 to 1 56m Normal Started pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Add eth0 [10.132.0.55/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Add eth0 [10.133.0.47/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-fb9597 Scaled up replica set payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc from 0 to 1 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc Error creating: pods "payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-fb9597: serviceaccount "payload-processing-e2e-worker-w6-fb9597" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc Created pod: payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h 56m Normal Pulling pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-1f3a6b Scaled up replica set payload-processing-e2e-worker-w0-1f3a6b-686494f97b from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-fb9597 Scaled up replica set payload-processing-e2e-worker-w6-fb9597-86489d9d58 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-1f3a6b-686494f97b Created pod: payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-f033b1 Scaled up replica set payload-pre-processing-e2e-worker-w2-f033b1-54b598887f from 0 to 1 56m Normal Started pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Container started 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b Created pod: payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt 56m Normal Created pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Container created 56m Normal Pulled pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Add eth0 [10.133.0.48/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f Error creating: pods "payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-f033b1: serviceaccount "payload-processing-e2e-worker-w2-f033b1" not found 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f Created pod: payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b Error creating: pods "payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-f033b1: serviceaccount "payload-processing-e2e-worker-w2-f033b1" not found 56m Normal Pulling pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Add eth0 [10.132.0.56/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-f033b1 Scaled up replica set payload-processing-e2e-worker-w2-f033b1-7f46dbb79b from 0 to 1 56m Normal Started pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Container started 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-0def36-55888856c8 Error creating: pods "payload-processing-e2e-worker-w4-0def36-55888856c8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-0def36: serviceaccount "payload-processing-e2e-worker-w4-0def36" not found 56m Normal Created pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Container created 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99 Created pod: payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb 56m Normal Pulled pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Add eth0 [10.133.0.49/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-53b784 Scaled up replica set payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99 from 0 to 1 56m Normal Pulling pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99 Error creating: pods "payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-53b784: serviceaccount "payload-processing-e2e-worker-w1-53b784" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-53b784 Scaled up replica set payload-processing-e2e-worker-w1-53b784-566d7f64fd from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-53b784-566d7f64fd Created pod: payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Add eth0 [10.133.0.50/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-53b784-566d7f64fd Error creating: pods "payload-processing-e2e-worker-w1-53b784-566d7f64fd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-53b784: serviceaccount "payload-processing-e2e-worker-w1-53b784" not found 56m Normal Pulling pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 56m Normal AddedInterface pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Add eth0 [10.132.0.57/23] from ovn-kubernetes 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c Error creating: pods "payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-0def36: serviceaccount "payload-processing-e2e-worker-w4-0def36" not found 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-0def36 Scaled up replica set payload-processing-e2e-worker-w4-0def36-55888856c8 from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-0def36-55888856c8 Created pod: payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g 56m Normal Started pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Container started 56m Normal Created pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Container created 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Add eth0 [10.132.0.58/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c Created pod: payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-0def36 Scaled up replica set payload-pre-processing-e2e-worker-w4-0def36-69fb87758c from 0 to 1 56m Normal Started pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Container started 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59 Created pod: payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r 56m Normal Started pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Container started 56m Normal Pulled pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Container started 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-ff3e70 Scaled up replica set payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59 from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-ff3e70 Scaled up replica set payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86 from 0 to 1 56m Normal Started pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Add eth0 [10.133.0.51/23] from ovn-kubernetes 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.657s (2.657s including waiting). Image size: 125353963 bytes. 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Container created 56m Normal Created pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Container created 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59 Error creating: pods "payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-ff3e70: serviceaccount "payload-processing-e2e-worker-w3-ff3e70" not found 56m Normal Created pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Container created 56m Normal AddedInterface pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Add eth0 [10.133.0.52/23] from ovn-kubernetes 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86 Created pod: payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 56m Normal Created pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Container created 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86 Error creating: pods "payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-ff3e70: serviceaccount "payload-processing-e2e-worker-w3-ff3e70" not found 56m Normal Started pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Container started 56m Normal Created pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Container created 56m Normal Pulled pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 814ms (814ms including waiting). Image size: 125353963 bytes. 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 781ms (781ms including waiting). Image size: 125353963 bytes. 56m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4 Error creating: pods "payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-b0b7a3: serviceaccount "payload-processing-e2e-worker-w5-b0b7a3" not found 56m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4 Created pod: payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 56m Normal Pulled pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c Error creating: pods "payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-b0b7a3: serviceaccount "payload-processing-e2e-worker-w5-b0b7a3" not found 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Created pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Container created 56m Normal Started pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Container started 56m Normal Started pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Add eth0 [10.133.0.53/23] from ovn-kubernetes 56m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-b0b7a3 Scaled up replica set payload-processing-e2e-worker-w5-b0b7a3-769756b4c4 from 0 to 1 56m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-b0b7a3 Scaled up replica set payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c from 0 to 1 56m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c Created pod: payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.797s (2.797s including waiting). Image size: 125353963 bytes. 56m Normal Created pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Container created 56m Normal AddedInterface pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Add eth0 [10.132.0.59/23] from ovn-kubernetes 56m Normal Created pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Container created 56m Normal Started pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Container started 56m Normal Started pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Container started 56m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Normal Started pod/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 Container started 56m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Add eth0 [10.132.0.60/23] from ovn-kubernetes 56m Normal AddedInterface pod/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 Add eth0 [10.132.0.61/23] from ovn-kubernetes 56m Normal Created pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Container created 56m Normal Created pod/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 Container created 56m Normal Pulled pod/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 56m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Readiness probe failed: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 56m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Readiness probe error: Get "http://10.134.0.38:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 56m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Container created 56m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Container started 56m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Pulled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Container started 56m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Container created 56m Normal Started pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Container started 56m Normal Created pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Container created 56m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Startup probe error: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Startup probe failed: Get "http://10.134.0.38:15021/healthz/ready": dial tcp 10.134.0.38:15021: connect: connection refused 56m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Container started 56m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Container created 56m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Container created 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Startup probe error: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused... 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Container started 56m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Container started 56m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Container created 56m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Container started 56m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Startup probe failed: Get "http://10.133.0.45:15021/healthz/ready": dial tcp 10.133.0.45:15021: connect: connection refused 56m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Container created 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Startup probe failed: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Startup probe error: Get "http://10.133.0.46:15021/healthz/ready": dial tcp 10.133.0.46:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Startup probe failed: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Startup probe error: Get "http://10.134.0.39:15021/healthz/ready": dial tcp 10.134.0.39:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Startup probe error: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused... 56m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Startup probe error: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 56m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Startup probe failed: Get "http://10.134.0.40:15021/healthz/ready": dial tcp 10.134.0.40:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 56m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Startup probe failed: Get "http://10.133.0.44:15021/healthz/ready": dial tcp 10.133.0.44:15021: connect: connection refused 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 55m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled up replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 1 to 2 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled up replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 1 to 2 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v 55m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Add eth0 [10.132.0.69/23] from ovn-kubernetes 55m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Add eth0 [10.133.0.60/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Container created 55m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Container started 55m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Container started 55m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Container created 55m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Startup probe failed: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Startup probe error: Get "http://10.132.0.69:15021/healthz/ready": dial tcp 10.132.0.69:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Startup probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Startup probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 55m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled up replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 1 to 2 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv 55m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Container created 55m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Add eth0 [10.132.0.70/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Container started 55m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled up replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 1 to 2 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w 55m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled up replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 1 to 2 55m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Add eth0 [10.132.0.75/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Container created 55m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Container started 55m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Container created 55m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Add eth0 [10.133.0.62/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Container started 54m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Startup probe failed: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Startup probe error: Get "http://10.133.0.62:15021/healthz/ready": dial tcp 10.133.0.62:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Startup probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Startup probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled up replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 1 to 2 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled up replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 2 to 3 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn 54m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Container created 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f 54m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Container created 54m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Container started 54m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Add eth0 [10.132.0.87/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Add eth0 [10.134.0.53/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Container started 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled up replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 2 to 3 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 54m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Add eth0 [10.134.0.51/23] from ovn-kubernetes 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled up replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 2 to 4 54m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Container started 54m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Container created 54m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Container started 54m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Add eth0 [10.134.0.52/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Container created 54m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Add eth0 [10.133.0.64/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Container created 54m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Container started 54m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Startup probe failed: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Startup probe error: Get "http://10.132.0.87:15021/healthz/ready": dial tcp 10.132.0.87:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Startup probe error: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": dial tcp 10.134.0.51:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Startup probe failed: Get "http://10.133.0.64:15021/healthz/ready": dial tcp 10.133.0.64:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Startup probe error: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Startup probe failed: Get "http://10.134.0.52:15021/healthz/ready": dial tcp 10.134.0.52:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Startup probe failed: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Startup probe error: Get "http://10.134.0.53:15021/healthz/ready": dial tcp 10.134.0.53:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Startup probe error: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Startup probe failed: Get "http://10.134.0.51:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 54m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Container created 54m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Add eth0 [10.133.0.67/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Container created 54m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Container started 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled up replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 3 to 4 54m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Add eth0 [10.132.0.91/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Container started 54m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Startup probe error: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Startup probe failed: Get "http://10.133.0.67:15021/healthz/ready": dial tcp 10.133.0.67:15021: connect: connection refused 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-f033b1-openshift-default Scaled up replica set e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 from 1 to 2 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled up replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 3 to 4 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 Created pod: e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 54m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Container started 54m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Container created 54m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Add eth0 [10.133.0.68/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Add eth0 [10.134.0.56/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Container created 54m Normal Started pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Container started 54m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Startup probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Startup probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled up replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 2 to 4 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled up replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 2 to 4 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled up replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 3 to 6 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 53m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Add eth0 [10.132.0.96/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Container created 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Add eth0 [10.132.0.97/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Container started 53m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Container created 53m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Add eth0 [10.133.0.70/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Add eth0 [10.134.0.61/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Container created 53m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Container started 53m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Container created 53m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Add eth0 [10.133.0.72/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz 53m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Add eth0 [10.133.0.71/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Container started 53m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Container created 53m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Add eth0 [10.132.0.98/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Container started 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled up replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 4 to 6 53m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Add eth0 [10.133.0.73/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Container created 53m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Container started 53m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Container created 53m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Add eth0 [10.132.0.95/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Container created 53m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Container started 53m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Container started 53m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Container started 53m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Add eth0 [10.134.0.59/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Container created 53m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Container started 53m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Container created 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled up replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 4 to 6 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Add eth0 [10.134.0.60/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Container started 53m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Container created 53m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Container started 53m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Startup probe error: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Startup probe failed: Get "http://10.134.0.60:15021/healthz/ready": dial tcp 10.134.0.60:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Startup probe error: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Startup probe failed: Get "http://10.134.0.61:15021/healthz/ready": dial tcp 10.134.0.61:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Startup probe error: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Startup probe failed: Get "http://10.133.0.71:15021/healthz/ready": dial tcp 10.133.0.71:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Startup probe error: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Startup probe failed: Get "http://10.133.0.72:15021/healthz/ready": dial tcp 10.133.0.72:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Startup probe failed: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Startup probe error: Get "http://10.132.0.97:15021/healthz/ready": dial tcp 10.132.0.97:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Startup probe error: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Startup probe failed: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Startup probe error: Get "http://10.132.0.96:15021/healthz/ready": dial tcp 10.132.0.96:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Startup probe failed: Get "http://10.132.0.95:15021/healthz/ready": dial tcp 10.132.0.95:15021: connect: connection refused 53m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 53m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-dsdmp 53m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled up replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 4 to 5 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Add eth0 [10.134.0.62/23] from ovn-kubernetes 53m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 53m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Container created 53m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Add eth0 [10.132.0.100/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Container created 53m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Container started 53m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Add eth0 [10.133.0.75/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn 53m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Container started 53m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Add eth0 [10.132.0.102/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 53m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Container created 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled up replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 6 to 8 53m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Container created 53m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Container started 53m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.163s (3.163s including waiting). Image size: 185476477 bytes. 53m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Readiness probe failed: HTTP probe failed with statuscode: 503 53m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Startup probe failed: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Startup probe error: Get "http://10.133.0.75:15021/healthz/ready": dial tcp 10.133.0.75:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Startup probe error: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Startup probe failed: Get "http://10.132.0.102:15021/healthz/ready": dial tcp 10.132.0.102:15021: connect: connection refused 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled up replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 4 to 8 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 Created pod: e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-f033b1-openshift-default Scaled up replica set e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 from 2 to 3 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Container created 53m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Container started 53m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Add eth0 [10.134.0.63/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Container created 53m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Container started 53m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Add eth0 [10.133.0.77/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Container created 53m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 53m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Add eth0 [10.132.0.110/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Container started 53m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Container created 53m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Add eth0 [10.132.0.111/23] from ovn-kubernetes 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 (combined from similar events): Created pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled up replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 6 to 10 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Add eth0 [10.132.0.112/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 (combined from similar events): Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Created pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Add eth0 [10.133.0.78/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Add eth0 [10.134.0.64/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Add eth0 [10.133.0.79/23] from ovn-kubernetes 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled up replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 6 to 10 53m Normal Created pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Container created 53m Normal AddedInterface pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Add eth0 [10.132.0.113/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Container created 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Container started 53m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Add eth0 [10.132.0.114/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Container started 53m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Container created 53m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Container created 53m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Add eth0 [10.134.0.65/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Container started 52m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Add eth0 [10.133.0.81/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Container created 52m Normal AddedInterface pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Add eth0 [10.133.0.80/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Container started 52m Normal Created pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Container created 52m Normal Pulled pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Container started 52m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Container started 52m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Container created 52m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Container created 52m Normal Started pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Container started 52m Normal Created pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Container created 52m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Container started 52m Normal Started pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Container started 52m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Startup probe error: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Startup probe failed: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Startup probe error: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Startup probe failed: Get "http://10.134.0.63:15021/healthz/ready": dial tcp 10.134.0.63:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Startup probe failed: Get "http://10.134.0.64:15021/healthz/ready": dial tcp 10.134.0.64:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Startup probe error: Get "http://10.134.0.65:15021/healthz/ready": dial tcp 10.134.0.65:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Startup probe failed: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Startup probe failed: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Startup probe error: Get "http://10.132.0.111:15021/healthz/ready": dial tcp 10.132.0.111:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Startup probe error: Get "http://10.132.0.110:15021/healthz/ready": dial tcp 10.132.0.110:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 52m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc 52m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 52m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled up replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 8 to 9 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 9; reason: 52m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 52m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Container started 52m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Add eth0 [10.134.0.66/23] from ovn-kubernetes 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Startup probe error: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Startup probe failed: Get "http://10.133.0.79:15021/healthz/ready": dial tcp 10.133.0.79:15021: connect: connection refused 52m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Container created 52m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Startup probe error: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Startup probe failed: Get "http://10.133.0.78:15021/healthz/ready": dial tcp 10.133.0.78:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Startup probe failed: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Startup probe error: Get "http://10.132.0.114:15021/healthz/ready": dial tcp 10.132.0.114:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled up replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 4 to 8 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g 52m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Add eth0 [10.132.0.122/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Container created 52m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Add eth0 [10.132.0.121/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Container started 52m Normal AddedInterface pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Add eth0 [10.133.0.83/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Add eth0 [10.134.0.67/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Container created 52m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Add eth0 [10.134.0.68/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Container started 52m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Container created 52m Normal Pulled pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Container started 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 Created pod: e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled up replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 8 to 10 52m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Container created 52m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Container started 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 (combined from similar events): Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Created pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 52m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Container started 52m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Container created 52m Normal AddedInterface pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Add eth0 [10.132.0.123/23] from ovn-kubernetes 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-f033b1-openshift-default Scaled up replica set e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 from 3 to 4 52m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Add eth0 [10.133.0.82/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Container created 52m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Container started 52m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Container started 52m Normal Created pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Container created 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Readiness probe failed: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Readiness probe error: Get "http://10.134.0.60:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Startup probe error: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Startup probe error: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Startup probe failed: Get "http://10.134.0.67:15021/healthz/ready": dial tcp 10.134.0.67:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Startup probe failed: Get "http://10.134.0.68:15021/healthz/ready": dial tcp 10.134.0.68:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Readiness probe failed: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Readiness probe error: Get "http://10.133.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Readiness probe error: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Readiness probe failed: Get "http://10.133.0.75:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": dial tcp 10.132.0.121:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Readiness probe failed: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Readiness probe error: Get "http://10.133.0.64:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Readiness probe error: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Readiness probe failed: Get "http://10.132.0.102:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Startup probe error: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Startup probe failed: Get "http://10.132.0.123:15021/healthz/ready": dial tcp 10.132.0.123:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Startup probe error: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Startup probe failed: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Startup probe failed: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Startup probe error: Get "http://10.132.0.122:15021/healthz/ready": dial tcp 10.132.0.122:15021: connect: connection refused... 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 9; reason: All metrics below target 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled up replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 8 to 9 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Container created 52m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Container started 52m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Add eth0 [10.133.0.85/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Startup probe failed: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Startup probe error: Get "http://10.133.0.85:15021/healthz/ready": dial tcp 10.133.0.85:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Readiness probe error: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Readiness probe error: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Readiness probe failed: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Readiness probe failed: Get "http://10.133.0.85:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Readiness probe failed: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Readiness probe error: Get "http://10.132.0.87:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Readiness probe failed: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Readiness probe error: Get "http://10.132.0.100:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulCreate replicaset/e2e-ait-13364262-openshift-default-84754cd9dc Created pod: e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf 51m Normal AddedLabel gateway/e2e-ait-681db477 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-681db477 51m Normal AddedLabel gateway/e2e-ait-13364262 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-13364262 51m Normal ScalingReplicaSet deployment/e2e-ait-13364262-openshift-default Scaled up replica set e2e-ait-13364262-openshift-default-84754cd9dc from 0 to 1 51m Normal ScalingReplicaSet deployment/e2e-ait-681db477-openshift-default Scaled up replica set e2e-ait-681db477-openshift-default-74959dc87f from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-ait-681db477-openshift-default-74959dc87f Created pod: e2e-ait-681db477-openshift-default-74959dc87f-fddv9 51m Normal AddedInterface pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Add eth0 [10.134.0.69/23] from ovn-kubernetes 51m Normal SuccessfulCreate replicaset/e2e-ait-93a347eb-openshift-default-95778769c Created pod: e2e-ait-93a347eb-openshift-default-95778769c-7g26j 51m Normal AddedLabel gateway/e2e-ait-93a347eb Added label istio.io/rev=openshift-gateway to gateway e2e-ait-93a347eb 51m Normal Started pod/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 Container started 51m Normal ScalingReplicaSet deployment/e2e-ait-93a347eb-openshift-default Scaled up replica set e2e-ait-93a347eb-openshift-default-95778769c from 0 to 1 51m Normal AddedInterface pod/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 Add eth0 [10.132.0.139/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 Container created 51m Normal AddedInterface pod/e2e-ait-93a347eb-openshift-default-95778769c-7g26j Add eth0 [10.133.0.86/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-ait-93a347eb-openshift-default-95778769c-7g26j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-ait-93a347eb-openshift-default-95778769c-7g26j Container created 51m Normal Started pod/e2e-ait-93a347eb-openshift-default-95778769c-7g26j Container started 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-13364262 Scaled up replica set payload-processing-e2e-ait-13364262-8f8d44b44 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-13364262-8f8d44b44 Created pod: payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-13364262-8f8d44b44 Error creating: pods "payload-processing-e2e-ait-13364262-8f8d44b44-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-13364262: serviceaccount "payload-processing-e2e-ait-13364262" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-13364262 Scaled up replica set payload-pre-processing-e2e-ait-13364262-55c96d8b7f from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-13364262-55c96d8b7f Created pod: payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-13364262-55c96d8b7f Error creating: pods "payload-pre-processing-e2e-ait-13364262-55c96d8b7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-13364262: serviceaccount "payload-processing-e2e-ait-13364262" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-681db477 Scaled up replica set payload-processing-e2e-ait-681db477-64946f668d from 0 to 1 51m Normal Started pod/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv Container started 51m Normal Created pod/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 Container created 51m Normal AddedInterface pod/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv Add eth0 [10.133.0.87/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 Container started 51m Normal Created pod/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv Container created 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-681db477-58c4bb787 Created pod: payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-681db477 Scaled up replica set payload-pre-processing-e2e-ait-681db477-58c4bb787 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-681db477-64946f668d Created pod: payload-processing-e2e-ait-681db477-64946f668d-mf6dm 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 Add eth0 [10.132.0.141/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-681db477-58c4bb787 Error creating: pods "payload-pre-processing-e2e-ait-681db477-58c4bb787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-681db477: serviceaccount "payload-processing-e2e-ait-681db477" not found 51m Normal Pulled pod/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-681db477-64946f668d Error creating: pods "payload-processing-e2e-ait-681db477-64946f668d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-681db477: serviceaccount "payload-processing-e2e-ait-681db477" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-93a347eb Scaled up replica set payload-processing-e2e-ait-93a347eb-67ff94dc58 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-93a347eb-67ff94dc58 Created pod: payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-93a347eb-67ff94dc58 Error creating: pods "payload-processing-e2e-ait-93a347eb-67ff94dc58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-93a347eb: serviceaccount "payload-processing-e2e-ait-93a347eb" not found 51m Normal AddedInterface pod/payload-processing-e2e-ait-681db477-64946f668d-mf6dm Add eth0 [10.133.0.88/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-ait-681db477-64946f668d-mf6dm Container started 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-93a347eb Scaled up replica set payload-pre-processing-e2e-ait-93a347eb-d9b77546c from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-93a347eb-d9b77546c Created pod: payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-93a347eb-d9b77546c Error creating: pods "payload-pre-processing-e2e-ait-93a347eb-d9b77546c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-93a347eb: serviceaccount "payload-processing-e2e-ait-93a347eb" not found 51m Normal Pulled pod/payload-processing-e2e-ait-681db477-64946f668d-mf6dm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr Add eth0 [10.132.0.142/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr Container started 51m Normal Created pod/payload-processing-e2e-ait-681db477-64946f668d-mf6dm Container created 51m Normal Created pod/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr Container created 51m Normal Pulled pod/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm Add eth0 [10.132.0.145/23] from ovn-kubernetes 51m Normal Created pod/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm Container created 51m Normal Started pod/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 Container started 51m Normal AddedInterface pod/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 Add eth0 [10.132.0.144/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm Container started 51m Normal Created pod/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 Container created 51m Normal Pulled pod/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-681db477-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-681db477-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx 51m Normal SuccessfulCreate replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc (combined from similar events): Created pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs 51m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled up replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 9 to 10 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 51m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled up replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 5 to 9 51m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg 51m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 51m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Container created 51m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Container started 51m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Add eth0 [10.132.0.148/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Container created 51m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Add eth0 [10.133.0.90/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Container started 51m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Container created 51m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Add eth0 [10.134.0.72/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Container created 51m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Add eth0 [10.134.0.73/23] from ovn-kubernetes 51m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Container started 51m Normal AddedInterface pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Add eth0 [10.132.0.149/23] from ovn-kubernetes 51m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Container started 51m Normal Started pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Container started 51m Normal Created pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Container created 51m Normal Pulled pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal ScalingReplicaSet deployment/e2e-migrate-1aee4437-gw-openshift-default Scaled up replica set e2e-migrate-1aee4437-gw-openshift-default-6795596c97 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-migrate-1aee4437-gw-openshift-default-6795596c97 Created pod: e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg 51m Normal AddedLabel gateway/e2e-migrate-1aee4437-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-1aee4437-gw 51m Normal AddedInterface pod/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg Add eth0 [10.132.0.151/23] from ovn-kubernetes 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Startup probe failed: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 51m Normal Pulled pod/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg Container created 51m Normal Started pod/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg Container started 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Startup probe error: Get "http://10.134.0.73:15021/healthz/ready": dial tcp 10.134.0.73:15021: connect: connection refused... 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Startup probe error: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Startup probe failed: Get "http://10.132.0.149:15021/healthz/ready": dial tcp 10.132.0.149:15021: connect: connection refused 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Startup probe error: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused... 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Startup probe failed: Get "http://10.132.0.148:15021/healthz/ready": dial tcp 10.132.0.148:15021: connect: connection refused 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-1aee4437 Scaled up replica set payload-processing-e2e-migrate-1aee4437-8649b8584d from 0 to 1 51m Warning FailedCreate replicaset/payload-processing-e2e-migrate-1aee4437-8649b8584d Error creating: pods "payload-processing-e2e-migrate-1aee4437-8649b8584d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-1aee4437: serviceaccount "payload-processing-e2e-migrate-1aee4437" not found 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c Created pod: payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-1aee4437 Scaled up replica set payload-pre-processing-e2e-migrate-1aee4437-867f875c6c from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c Error creating: pods "payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-1aee4437: serviceaccount "payload-processing-e2e-migrate-1aee4437" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-1aee4437-8649b8584d Created pod: payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-681db477-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Readiness probe failed: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-13364262-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Readiness probe error: Get "http://10.132.0.96:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-13364262-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-681db477-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 4 51m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Readiness probe failed: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Readiness probe error: Get "http://10.132.0.121:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-782nd 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-8dg8l 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-93a347eb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-93a347eb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 Add eth0 [10.133.0.91/23] from ovn-kubernetes 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Add eth0 [10.133.0.93/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Container created 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Container started 51m Normal AddedInterface pod/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm Add eth0 [10.133.0.92/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 Container started 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 51m Normal Pulled pod/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 Container created 51m Normal Started pod/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm Container started 51m Normal Created pod/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm Container created 51m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Readiness probe error: HTTP probe failed with statuscode: 503... 51m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Add eth0 [10.132.0.155/23] from ovn-kubernetes 51m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Readiness probe failed: HTTP probe failed with statuscode: 503 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.914s (3.914s including waiting). Image size: 185476477 bytes. 51m Normal Killing pod/payload-pre-processing-e2e-ait-681db477-58c4bb787-pnsjr Stopping container payload-pre-processing 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Container created 51m Normal Killing pod/payload-processing-e2e-ait-681db477-64946f668d-mf6dm Stopping container payload-processing 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Container started 51m Normal Killing pod/payload-processing-e2e-ait-13364262-8f8d44b44-ndwvv Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-ait-13364262-55c96d8b7f-mktk7 Stopping container payload-pre-processing 51m Normal Killing pod/payload-processing-e2e-ait-93a347eb-67ff94dc58-f7rb9 Stopping container payload-processing 51m Normal Killing pod/payload-pre-processing-e2e-ait-93a347eb-d9b77546c-qdnhm Stopping container payload-pre-processing 51m Normal Killing pod/e2e-ait-93a347eb-openshift-default-95778769c-7g26j Stopping container istio-proxy 51m Normal Started pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Container started 51m Normal Created pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Container created 51m Normal Pulled pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-13364262-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-13364262-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning Unhealthy pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Startup probe failed: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused 51m Warning ProbeError pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Startup probe error: Get "http://10.134.0.69:15021/healthz/ready": dial tcp 10.134.0.69:15021: connect: connection refused... 50m Normal Killing pod/e2e-ait-681db477-openshift-default-74959dc87f-fddv9 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-derive-d5713372-openshift-default Scaled up replica set e2e-derive-d5713372-openshift-default-5ddf58bc89 from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-derive-d5713372-openshift-default-5ddf58bc89 Created pod: e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh 50m Normal AddedLabel gateway/e2e-derive-d5713372 Added label istio.io/rev=openshift-gateway to gateway e2e-derive-d5713372 50m Warning FailedToRetrieveImagePullSecret pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Unable to retrieve some image pull secrets (e2e-ait-13364262-openshift-default-dockercfg-hk9r6); attempting to pull the image may not succeed. 50m Normal Created pod/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh Container created 50m Normal Pulled pod/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Killing pod/e2e-ait-13364262-openshift-default-84754cd9dc-tnjlf Stopping container istio-proxy 50m Normal Started pod/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh Container started 50m Normal AddedInterface pod/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh Add eth0 [10.134.0.75/23] from ovn-kubernetes 50m Warning FailedCreate replicaset/payload-processing-e2e-derive-d5713372-5555895f6 Error creating: pods "payload-processing-e2e-derive-d5713372-5555895f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-d5713372: serviceaccount "payload-processing-e2e-derive-d5713372" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-d5713372-5555895f6 Created pod: payload-processing-e2e-derive-d5713372-5555895f6-lk42f 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-d5713372 Scaled up replica set payload-pre-processing-e2e-derive-d5713372-57cf5fd967 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-d5713372 Scaled up replica set payload-processing-e2e-derive-d5713372-5555895f6 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-d5713372-57cf5fd967 Created pod: payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-d5713372-57cf5fd967 Error creating: pods "payload-pre-processing-e2e-derive-d5713372-57cf5fd967-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-d5713372: serviceaccount "payload-processing-e2e-derive-d5713372" not found 50m Normal Pulled pod/payload-processing-e2e-derive-d5713372-5555895f6-lk42f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-processing-e2e-derive-d5713372-5555895f6-lk42f Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 Add eth0 [10.133.0.94/23] from ovn-kubernetes 50m Normal Started pod/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 Container started 50m Normal AddedInterface pod/payload-processing-e2e-derive-d5713372-5555895f6-lk42f Add eth0 [10.133.0.95/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 Container created 50m Normal Created pod/payload-processing-e2e-derive-d5713372-5555895f6-lk42f Container created 50m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled up replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 9 to 10 50m Normal SuccessfulCreate replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 (combined from similar events): Created pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb 50m Normal AddedInterface pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Add eth0 [10.134.0.78/23] from ovn-kubernetes 50m Normal Pulled pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Started pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Container started 50m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 50m Normal Created pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Container created 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-d5713372-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-d5713372-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-1aee4437-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-1aee4437-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 50m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Readiness probe failed: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Normal Killing pod/payload-processing-e2e-migrate-1aee4437-8649b8584d-mq9rm Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-migrate-1aee4437-867f875c6c-vdcf6 Stopping container payload-pre-processing 50m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Readiness probe error: Get "http://10.133.0.62:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning ProbeError pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Startup probe error: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused... 50m Warning Unhealthy pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Startup probe failed: Get "http://10.134.0.78:15021/healthz/ready": dial tcp 10.134.0.78:15021: connect: connection refused 50m Normal Killing pod/e2e-migrate-1aee4437-gw-openshift-default-6795596c97-zz4dg Stopping container istio-proxy 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-d5713372-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-d5713372-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-1783e3-openshift-default Scaled up replica set e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh 50m Normal AddedLabel gateway/e2e-shared-a-w4-1783e3 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-1783e3 50m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Add eth0 [10.134.0.79/23] from ovn-kubernetes 50m Normal Killing pod/payload-pre-processing-e2e-derive-d5713372-57cf5fd967-rnrv2 Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-derive-d5713372-5555895f6-lk42f Stopping container payload-processing 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-1783e3 Scaled up replica set payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9 from 0 to 1 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-1783e3 Scaled up replica set payload-processing-e2e-shared-a-w4-1783e3-58446fb578 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-1783e3-58446fb578 Created pod: payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9 Error creating: pods "payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-1783e3: serviceaccount "payload-processing-e2e-shared-a-w4-1783e3" not found 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9 Created pod: payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-1783e3-58446fb578 Error creating: pods "payload-processing-e2e-shared-a-w4-1783e3-58446fb578-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-1783e3: serviceaccount "payload-processing-e2e-shared-a-w4-1783e3" not found 50m Normal Started pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Add eth0 [10.134.0.80/23] from ovn-kubernetes 50m Normal Pulling pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 50m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Add eth0 [10.133.0.99/23] from ovn-kubernetes 50m Normal Pulled pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Created pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Container created 50m Normal Killing pod/e2e-derive-d5713372-openshift-default-5ddf58bc89-xmgvh Stopping container istio-proxy 50m Normal AddedLabel gateway/e2e-shared-a-w5-f758d9 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-f758d9 50m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-f758d9-openshift-default Scaled up replica set e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f from 0 to 1 50m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f Created pod: e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw 50m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Container created 50m Normal AddedInterface pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Add eth0 [10.134.0.81/23] from ovn-kubernetes 50m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.386s (3.386s including waiting). Image size: 125353963 bytes. 50m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Container started 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799 Created pod: payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-f758d9 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799 from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-f758d9: serviceaccount "payload-processing-e2e-shared-a-w5-f758d9" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-f758d9 Scaled up replica set payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84 Created pod: payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84 Error creating: pods "payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-f758d9: serviceaccount "payload-processing-e2e-shared-a-w5-f758d9" not found 50m Normal Pulled pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Container started 50m Normal Created pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Container created 50m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Add eth0 [10.134.0.83/23] from ovn-kubernetes 50m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Add eth0 [10.133.0.100/23] from ovn-kubernetes 50m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Container created 50m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Container started 49m Normal AddedLabel gateway/e2e-shared-b-w4-e2bf63 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-e2bf63 49m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-e2bf63-openshift-default Scaled up replica set e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 from 0 to 1 49m Normal AddedInterface pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Add eth0 [10.132.0.163/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 Created pod: e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p 49m Normal AddedLabel gateway/e2e-shared-b-w5-2c5135 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-2c5135 49m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89 Created pod: e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw 49m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-2c5135-openshift-default Scaled up replica set e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89 from 0 to 1 49m Normal AddedInterface pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Add eth0 [10.133.0.101/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-e2bf63 Scaled up replica set payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f from 0 to 1 49m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Add eth0 [10.134.0.85/23] from ovn-kubernetes 49m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b Error creating: pods "payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-e2bf63: serviceaccount "payload-processing-e2e-shared-b-w4-e2bf63" not found 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f Created pod: payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b Created pod: payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f Error creating: pods "payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-e2bf63: serviceaccount "payload-processing-e2e-shared-b-w4-e2bf63" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-e2bf63 Scaled up replica set payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b from 0 to 1 49m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Add eth0 [10.133.0.102/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Container started 49m Normal Started pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Container started 49m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Container created 49m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Container created 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99 Created pod: payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99 Error creating: pods "payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-2c5135: serviceaccount "payload-processing-e2e-shared-b-w5-2c5135" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4 Created pod: payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj 49m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4 Error creating: pods "payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-2c5135: serviceaccount "payload-processing-e2e-shared-b-w5-2c5135" not found 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-2c5135 Scaled up replica set payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99 from 0 to 1 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-2c5135 Scaled up replica set payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4 from 0 to 1 49m Normal Created pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Container created 49m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Container started 49m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Add eth0 [10.134.0.86/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Add eth0 [10.132.0.165/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Container started 49m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Container created 49m Normal ScalingReplicaSet deployment/e2e-mt-21dd16de-openshift-default Scaled up replica set e2e-mt-21dd16de-openshift-default-5db99d957 from 0 to 1 49m Normal AddedLabel gateway/e2e-mt-21dd16de Added label istio.io/rev=openshift-gateway to gateway e2e-mt-21dd16de 49m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 49m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Add eth0 [10.132.0.166/23] from ovn-kubernetes 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-21dd16de-5b6fffb6c4 Created pod: payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 49m Normal Pulled pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-21dd16de Scaled up replica set payload-pre-processing-e2e-mt-21dd16de-568bb59d5f from 0 to 1 49m Normal Created pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Container created 49m Normal Started pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Container started 49m Warning FailedCreate replicaset/payload-processing-e2e-mt-21dd16de-5b6fffb6c4 Error creating: pods "payload-processing-e2e-mt-21dd16de-5b6fffb6c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-21dd16de: serviceaccount "payload-processing-e2e-mt-21dd16de" not found 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f Error creating: pods "payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-21dd16de: serviceaccount "payload-processing-e2e-mt-21dd16de" not found 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-21dd16de Scaled up replica set payload-processing-e2e-mt-21dd16de-5b6fffb6c4 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f Created pod: payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln 49m Normal Started pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Container started 49m Normal AddedInterface pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Add eth0 [10.134.0.88/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Container started 49m Normal Created pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Container created 49m Normal Created pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Container created 49m Normal AddedInterface pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Add eth0 [10.133.0.104/23] from ovn-kubernetes 49m Normal Pulled pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal Pulled pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-f758d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-f758d9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning ProbeError pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 49m Warning Unhealthy pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Readiness probe error: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Readiness probe failed: Get "http://10.132.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Warning Unhealthy pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Readiness probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 48m Normal Pulled pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Warning ProbeError pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Readiness probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 48m Normal Started pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Container started 48m Normal Pulled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Container created 48m Normal Started pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Container started 48m Normal Created pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Container created 48m Warning Unhealthy pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Startup probe failed: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Startup probe error: Get "http://10.132.0.163:15021/healthz/ready": dial tcp 10.132.0.163:15021: connect: connection refused... 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-2c5135-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-2c5135-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-e2bf63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-e2bf63-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-2c5135-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-2c5135-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Readiness probe error: Get "http://10.132.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 48m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Readiness probe failed: Get "http://10.132.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 48m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 48m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Container created 48m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Container started 48m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Startup probe error: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Startup probe failed: Get "http://10.132.0.166:15021/healthz/ready": dial tcp 10.132.0.166:15021: connect: connection refused 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-21dd16de-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal ScalingReplicaSet deployment/e2e-mt-b1b5598d-openshift-default Scaled up replica set e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c from 0 to 1 48m Normal AddedLabel gateway/e2e-mt-b1b5598d Added label istio.io/rev=openshift-gateway to gateway e2e-mt-b1b5598d 48m Normal SuccessfulCreate replicaset/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c Created pod: e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn 48m Normal AddedInterface pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Add eth0 [10.132.0.172/23] from ovn-kubernetes 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-21dd16de-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5 Created pod: payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x 48m Normal Pulled pod/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-b1b5598d-684b864db8 Created pod: payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 48m Normal AddedInterface pod/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x Add eth0 [10.134.0.90/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-b1b5598d Scaled up replica set payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5 from 0 to 1 48m Warning FailedCreate replicaset/payload-processing-e2e-mt-b1b5598d-684b864db8 Error creating: pods "payload-processing-e2e-mt-b1b5598d-684b864db8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-b1b5598d: serviceaccount "payload-processing-e2e-mt-b1b5598d" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-b1b5598d Scaled up replica set payload-processing-e2e-mt-b1b5598d-684b864db8 from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5 Error creating: pods "payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-b1b5598d: serviceaccount "payload-processing-e2e-mt-b1b5598d" not found 48m Normal Started pod/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x Container started 48m Normal Created pod/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x Container created 48m Normal Created pod/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 Container created 48m Normal Started pod/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 Container started 48m Normal Pulled pod/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 Add eth0 [10.133.0.105/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled down replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 9 to 8 48m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld 48m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Stopping container istio-proxy 48m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 8; reason: All metrics below target 47m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-j57ld Readiness probe error: HTTP probe failed with statuscode: 503... 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-21dd16de-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-21dd16de-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-b1b5598d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-b1b5598d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled down replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 8 to 6 47m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl 47m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-78nhl Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-sbgs7 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 6; reason: All metrics below target 47m Warning ProbeError pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Readiness probe error: Get "http://10.132.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Readiness probe failed: Get "http://10.132.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Created pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Container created 47m Normal Pulled pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Container started 47m Normal SuccessfulDelete replicaset/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 Deleted pod: e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 47m Normal Killing pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-ks876 Stopping container istio-proxy 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default New size: 3; reason: All metrics below target 47m Normal ScalingReplicaSet deployment/e2e-worker-w2-f033b1-openshift-default Scaled down replica set e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 from 4 to 3 47m Normal ScalingReplicaSet deployment/e2e-api-a-d506c6-openshift-default Scaled up replica set e2e-api-a-d506c6-openshift-default-55bb48b9b5 from 0 to 1 47m Normal AddedLabel gateway/e2e-api-a-d506c6 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-d506c6 47m Normal SuccessfulCreate replicaset/e2e-api-a-d506c6-openshift-default-55bb48b9b5 Created pod: e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls 47m Warning ProbeError pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Startup probe error: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Startup probe failed: Get "http://10.132.0.172:15021/healthz/ready": dial tcp 10.132.0.172:15021: connect: connection refused 47m Normal AddedInterface pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Add eth0 [10.132.0.175/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-d506c6 Scaled up replica set payload-pre-processing-e2e-api-a-d506c6-9698d4dc from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-d506c6-9698d4dc Created pod: payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 47m Warning FailedCreate replicaset/payload-processing-e2e-api-a-d506c6-64b85f4d8b Error creating: pods "payload-processing-e2e-api-a-d506c6-64b85f4d8b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d506c6: serviceaccount "payload-processing-e2e-api-a-d506c6" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-d506c6-64b85f4d8b Created pod: payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-d506c6 Scaled up replica set payload-processing-e2e-api-a-d506c6-64b85f4d8b from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-d506c6-9698d4dc Error creating: pods "payload-pre-processing-e2e-api-a-d506c6-9698d4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-d506c6: serviceaccount "payload-processing-e2e-api-a-d506c6" not found 47m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 Add eth0 [10.133.0.106/23] from ovn-kubernetes 47m Normal Created pod/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 Container created 47m Normal Pulled pod/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw Container started 47m Normal Started pod/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 Container started 47m Normal Created pod/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw Container created 47m Normal Pulled pod/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw Add eth0 [10.133.0.107/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/e2e-api-a-a19582-openshift-default Scaled up replica set e2e-api-a-a19582-openshift-default-7df57db669 from 0 to 1 47m Normal AddedInterface pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Add eth0 [10.133.0.108/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/e2e-api-a-a19582-openshift-default-7df57db669 Created pod: e2e-api-a-a19582-openshift-default-7df57db669-fzn8r 47m Normal AddedLabel gateway/e2e-api-a-a19582 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-a19582 47m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled down replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 10 to 9 47m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk Stopping container istio-proxy 47m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-n7zmk 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 9; reason: All metrics below target 46m Normal Killing pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Stopping container payload-pre-processing 46m Warning FailedCreate replicaset/payload-processing-e2e-api-a-a19582-758fdc7b46 Error creating: pods "payload-processing-e2e-api-a-a19582-758fdc7b46-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a19582: serviceaccount "payload-processing-e2e-api-a-a19582" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-a19582 Scaled up replica set payload-pre-processing-e2e-api-a-a19582-85fd78cb9f from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-a19582 Scaled up replica set payload-processing-e2e-api-a-a19582-758fdc7b46 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-a19582-758fdc7b46 Created pod: payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f Error creating: pods "payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-a19582: serviceaccount "payload-processing-e2e-api-a-a19582" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f Created pod: payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv 46m Normal Created pod/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx Container created 46m Normal Pulled pod/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv Container created 46m Normal Started pod/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv Add eth0 [10.134.0.92/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv Container started 46m Normal AddedInterface pod/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx Add eth0 [10.134.0.93/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-b1b5598d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-b1b5598d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal ScalingReplicaSet deployment/e2e-inf-a-edd32a-openshift-default Scaled up replica set e2e-inf-a-edd32a-openshift-default-7cf5f49d65 from 0 to 1 46m Normal AddedLabel gateway/e2e-inf-a-edd32a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-edd32a 46m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq 46m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Add eth0 [10.132.0.179/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55 Error creating: pods "payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-edd32a: serviceaccount "payload-processing-e2e-inf-a-edd32a" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-edd32a-76686b7d6 Created pod: payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-edd32a Scaled up replica set payload-processing-e2e-inf-a-edd32a-76686b7d6 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-edd32a-76686b7d6 Error creating: pods "payload-processing-e2e-inf-a-edd32a-76686b7d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-edd32a: serviceaccount "payload-processing-e2e-inf-a-edd32a" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55 Created pod: payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-edd32a Scaled up replica set payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55 from 0 to 1 46m Normal Created pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Container created 46m Normal AddedInterface pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Add eth0 [10.134.0.95/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Container started 46m Normal Pulled pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Add eth0 [10.133.0.110/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Container created 46m Warning Unhealthy pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-4djln Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 46m Normal ScalingReplicaSet deployment/e2e-api-b-004180-openshift-default Scaled up replica set e2e-api-b-004180-openshift-default-7dfb78476f from 0 to 1 46m Warning Unhealthy pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-4j885 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 46m Normal SuccessfulCreate replicaset/e2e-api-b-004180-openshift-default-7dfb78476f Created pod: e2e-api-b-004180-openshift-default-7dfb78476f-25d2k 46m Normal AddedLabel gateway/e2e-api-b-004180 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-004180 46m Normal AddedInterface pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Add eth0 [10.132.0.180/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-processing-e2e-api-b-004180-548d486474 Error creating: pods "payload-processing-e2e-api-b-004180-548d486474-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-004180: serviceaccount "payload-processing-e2e-api-b-004180" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-004180 Scaled up replica set payload-pre-processing-e2e-api-b-004180-845f96955 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-004180-845f96955 Created pod: payload-pre-processing-e2e-api-b-004180-845f96955-lw94h 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-004180-845f96955 Error creating: pods "payload-pre-processing-e2e-api-b-004180-845f96955-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-004180: serviceaccount "payload-processing-e2e-api-b-004180" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-004180 Scaled up replica set payload-processing-e2e-api-b-004180-548d486474 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-004180-548d486474 Created pod: payload-processing-e2e-api-b-004180-548d486474-4m87j 46m Normal Started pod/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h Container started 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb 46m Normal AddedInterface pod/payload-processing-e2e-api-b-004180-548d486474-4m87j Add eth0 [10.134.0.97/23] from ovn-kubernetes 46m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-6sm24 Stopping container istio-proxy 46m Normal Pulled pod/payload-processing-e2e-api-b-004180-548d486474-4m87j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-api-b-004180-548d486474-4m87j Container created 46m Normal Started pod/payload-processing-e2e-api-b-004180-548d486474-4m87j Container started 46m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled down replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 10 to 7 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts 46m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-4zzts Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Stopping container istio-proxy 46m Normal Created pod/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h Container created 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 7; reason: All metrics below target 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 8; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled down replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 9 to 8 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h Add eth0 [10.134.0.96/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-ph8sb Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-l2c57 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal SuccessfulCreate replicaset/e2e-api-b-0390e1-openshift-default-5499d6bdcf Created pod: e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 46m Normal ScalingReplicaSet deployment/e2e-api-b-0390e1-openshift-default Scaled up replica set e2e-api-b-0390e1-openshift-default-5499d6bdcf from 0 to 1 46m Normal AddedLabel gateway/e2e-api-b-0390e1 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-0390e1 46m Normal AddedInterface pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Add eth0 [10.134.0.98/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5 Created pod: payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5 Error creating: pods "payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0390e1: serviceaccount "payload-processing-e2e-api-b-0390e1" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-0390e1 Scaled up replica set payload-processing-e2e-api-b-0390e1-7b5b74b5c6 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-api-b-0390e1-7b5b74b5c6 Error creating: pods "payload-processing-e2e-api-b-0390e1-7b5b74b5c6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-0390e1: serviceaccount "payload-processing-e2e-api-b-0390e1" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-0390e1 Scaled up replica set payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-0390e1-7b5b74b5c6 Created pod: payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr 46m Normal Pulled pod/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-21dd16de Scaled up replica set payload-pre-processing-e2e-mt-21dd16de-568bb59d5f from 0 to 1 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts Add eth0 [10.133.0.112/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts Container created 46m Normal Started pod/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts Container started 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-21dd16de Scaled up replica set payload-processing-e2e-mt-21dd16de-5b6fffb6c4 from 0 to 1 46m Normal Started pod/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr Container started 46m Warning FailedCreate replicaset/payload-processing-e2e-mt-21dd16de-5b6fffb6c4 Error creating: pods "payload-processing-e2e-mt-21dd16de-5b6fffb6c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-21dd16de: serviceaccount "payload-processing-e2e-mt-21dd16de" not found 46m Normal AddedInterface pod/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr Add eth0 [10.133.0.113/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f Error creating: pods "payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-21dd16de: serviceaccount "payload-processing-e2e-mt-21dd16de" not found 46m Normal Created pod/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr Container created 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-21dd16de-5b6fffb6c4 Created pod: payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f Created pod: payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r 46m Normal AddedInterface pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd Add eth0 [10.133.0.115/23] from ovn-kubernetes 46m Normal Started pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r Container started 46m Normal Created pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r Container created 46m Normal Pulled pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r Add eth0 [10.133.0.114/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd Container started 46m Normal Created pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd Container created 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 7; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv 46m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-fbrkv Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default Scaled down replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 8 to 7 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-702002-openshift-default Scaled up replica set e2e-shared-a-w2-702002-openshift-default-5588c5c5d9 from 0 to 1 46m Normal AddedLabel gateway/e2e-shared-a-w2-702002 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-702002 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9 Created pod: e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt 46m Normal AddedInterface pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Add eth0 [10.132.0.181/23] from ovn-kubernetes 46m Normal SuccessfulCreate replicaset/e2e-inf-b-3577a0-openshift-default-84498fd47 Created pod: e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv 46m Normal ScalingReplicaSet deployment/e2e-inf-b-3577a0-openshift-default Scaled up replica set e2e-inf-b-3577a0-openshift-default-84498fd47 from 0 to 1 46m Normal AddedLabel gateway/e2e-inf-b-3577a0 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-3577a0 46m Normal AddedInterface pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Add eth0 [10.132.0.183/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-702002 Scaled up replica set payload-processing-e2e-shared-a-w2-702002-768fc75ff4 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-702002 Scaled up replica set payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7 from 0 to 1 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Readiness probe failed: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-702002-768fc75ff4 Error creating: pods "payload-processing-e2e-shared-a-w2-702002-768fc75ff4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-702002: serviceaccount "payload-processing-e2e-shared-a-w2-702002" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7 Created pod: payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-702002-768fc75ff4 Created pod: payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-702002: serviceaccount "payload-processing-e2e-shared-a-w2-702002" not found 46m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Readiness probe error: Get "http://10.134.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Readiness probe failed: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Readiness probe error: Get "http://10.134.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Add eth0 [10.134.0.99/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Container created 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-3577a0-d87647984 Created pod: payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-3577a0 Scaled up replica set payload-processing-e2e-inf-b-3577a0-d87647984 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc Created pod: payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc Error creating: pods "payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3577a0: serviceaccount "payload-processing-e2e-inf-b-3577a0" not found 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-3577a0 Scaled up replica set payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-3577a0-d87647984 Error creating: pods "payload-processing-e2e-inf-b-3577a0-d87647984-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3577a0: serviceaccount "payload-processing-e2e-inf-b-3577a0" not found 45m Normal Created pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Container created 45m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Add eth0 [10.133.0.119/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Add eth0 [10.134.0.101/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Pulled pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Container created 45m Normal AddedInterface pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Add eth0 [10.134.0.100/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Container started 45m Normal Started pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Container started 45m Normal Pulled pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Container started 45m Normal Created pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Container created 45m Normal Killing pod/payload-pre-processing-e2e-mt-b1b5598d-5779c6d5c5-lrj7x Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-mt-b1b5598d-684b864db8-46b66 Stopping container payload-processing 45m Normal Created pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Container created 45m Normal Pulled pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Container started 45m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz 45m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled down replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 7 to 6 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 6; reason: All metrics below target 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-bgwd9 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default New size: 6; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-hqdjz Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Startup probe failed: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused 45m Warning ProbeError pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Startup probe error: Get "http://10.133.0.108:15021/healthz/ready": dial tcp 10.133.0.108:15021: connect: connection refused... 45m Normal Killing pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Stopping container istio-proxy 45m Normal AddedLabel gateway/e2e-api-a-bb882c Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-bb882c 45m Normal ScalingReplicaSet deployment/e2e-api-a-bb882c-openshift-default Scaled up replica set e2e-api-a-bb882c-openshift-default-64f5fbfd9f from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-api-a-bb882c-openshift-default-64f5fbfd9f Created pod: e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz 45m Warning Unhealthy pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-mt-b1b5598d-openshift-default-6ddb6cbb5c-xqmkn Readiness probe error: HTTP probe failed with statuscode: 503... 45m Normal AddedInterface pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Add eth0 [10.132.0.186/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-702002-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-702002-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987 Created pod: payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-bb882c-8486dc96fb Created pod: payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-bb882c Scaled up replica set payload-processing-e2e-api-a-bb882c-8486dc96fb from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-bb882c Scaled up replica set payload-pre-processing-e2e-api-a-bb882c-c9cf8d987 from 0 to 1 45m Warning FailedCreate replicaset/payload-processing-e2e-api-a-bb882c-8486dc96fb Error creating: pods "payload-processing-e2e-api-a-bb882c-8486dc96fb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-bb882c: serviceaccount "payload-processing-e2e-api-a-bb882c" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987 Error creating: pods "payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-bb882c: serviceaccount "payload-processing-e2e-api-a-bb882c" not found 45m Normal Created pod/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g Container created 45m Normal Killing pod/payload-processing-e2e-api-a-d506c6-64b85f4d8b-5xsmw Stopping container payload-processing 45m Normal Pulled pod/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g Container started 45m Normal Created pod/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj Container created 45m Normal Pulled pod/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g Add eth0 [10.133.0.120/23] from ovn-kubernetes 45m Normal Killing pod/payload-pre-processing-e2e-api-a-d506c6-9698d4dc-46cl5 Stopping container payload-pre-processing 45m Normal Started pod/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj Container started 45m Normal AddedInterface pod/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj Add eth0 [10.134.0.104/23] from ovn-kubernetes 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a19582-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a19582-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-pre-processing-e2e-mt-21dd16de-568bb59d5f-hs94r Stopping container payload-pre-processing 45m Normal Killing pod/payload-processing-e2e-mt-21dd16de-5b6fffb6c4-db9jd Stopping container payload-processing 45m Normal Killing pod/payload-processing-e2e-api-a-a19582-758fdc7b46-l7zwx Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-api-a-a19582-85fd78cb9f-wf4xv Stopping container payload-pre-processing 45m Warning ProbeError pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Readiness probe error: Get "http://10.132.0.175:15021/healthz/ready": read tcp 10.132.0.2:58972->10.132.0.175:15021: read: connection reset by peer... 45m Warning Unhealthy pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Readiness probe failed: Get "http://10.132.0.175:15021/healthz/ready": read tcp 10.132.0.2:58972->10.132.0.175:15021: read: connection reset by peer 45m Normal Pulled pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Container created 45m Normal Started pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Container started 45m Normal AddedLabel gateway/e2e-shared-b-w2-de0cba Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-de0cba 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-de0cba-openshift-default Scaled up replica set e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7 from 0 to 1 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7 Created pod: e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-a19582-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-a19582-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal AddedInterface pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Add eth0 [10.133.0.122/23] from ovn-kubernetes 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 9; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled down replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 10 to 7 45m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 7; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled down replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 10 to 9 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 45m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-j9vlq Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 8; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled down replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 10 to 8 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-m2hqb 45m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fmpxs Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-b2bc6 Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-6mzs4 Stopping container istio-proxy 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c Error creating: pods "payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-de0cba: serviceaccount "payload-processing-e2e-shared-b-w2-de0cba" not found 45m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Readiness probe failed: HTTP probe failed with statuscode: 503 45m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-zvw97 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4 Error creating: pods "payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-de0cba: serviceaccount "payload-processing-e2e-shared-b-w2-de0cba" not found 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-de0cba Scaled up replica set payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-de0cba Scaled up replica set payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4 from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4 Created pod: payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c Created pod: payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl 45m Normal Started pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Container started 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d506c6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Container started 45m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning Unhealthy pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 45m Warning ProbeError pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Add eth0 [10.134.0.106/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Add eth0 [10.134.0.105/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d506c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Pulled pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Container created 45m Normal Created pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Container created 45m Warning Unhealthy pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Readiness probe failed: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Readiness probe error: Get "http://10.134.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal Created pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Container created 45m Normal Pulled pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Container started 45m Warning Unhealthy pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Readiness probe failed: Get "http://10.132.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Readiness probe error: Get "http://10.132.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Readiness probe failed: Get "http://10.132.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Readiness probe error: Get "http://10.132.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal SuccessfulCreate replicaset/e2e-api-b-18657f-openshift-default-54fcbd9989 Created pod: e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz 45m Normal Pulled pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled down replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 8 to 7 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 7; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-api-b-18657f-openshift-default Scaled up replica set e2e-api-b-18657f-openshift-default-54fcbd9989 from 0 to 1 45m Normal AddedLabel gateway/e2e-api-b-18657f Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-18657f 45m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled down replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 9 to 7 45m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-2tzsx 45m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-zzffg Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 7; reason: All metrics below target 45m Normal Pulled pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Container created 45m Normal Started pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Container started 45m Normal Created pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Container created 45m Warning Unhealthy pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Startup probe failed: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused 45m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Stopping container istio-proxy 45m Warning ProbeError pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Startup probe error: Get "http://10.134.0.98:15021/healthz/ready": dial tcp 10.134.0.98:15021: connect: connection refused... 45m Normal Started pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Container started 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf 45m Normal AddedInterface pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Add eth0 [10.132.0.191/23] from ovn-kubernetes 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-d506c6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-d506c6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-psclf Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-18657f Scaled up replica set payload-processing-e2e-api-b-18657f-865b44ff74 from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f Error creating: pods "payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-18657f: serviceaccount "payload-processing-e2e-api-b-18657f" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-18657f Scaled up replica set payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-18657f-865b44ff74 Created pod: payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 44m Warning FailedCreate replicaset/payload-processing-e2e-api-b-18657f-865b44ff74 Error creating: pods "payload-processing-e2e-api-b-18657f-865b44ff74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-18657f: serviceaccount "payload-processing-e2e-api-b-18657f" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f Created pod: payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd 44m Normal Pulled pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 Container created 44m Normal Started pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Container started 44m Normal Pulled pod/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Container created 44m Normal Started pod/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 Container started 44m Normal AddedInterface pod/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 Add eth0 [10.133.0.124/23] from ovn-kubernetes 44m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Add eth0 [10.134.0.107/23] from ovn-kubernetes 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-21dd16de-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 44m Normal Killing pod/e2e-api-a-d506c6-openshift-default-55bb48b9b5-8k5ls Stopping container istio-proxy 44m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp 44m Normal ScalingReplicaSet deployment/e2e-mt-21dd16de-openshift-default Scaled up replica set e2e-mt-21dd16de-openshift-default-5db99d957 from 1 to 4 44m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg 44m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr 44m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Container created 44m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Add eth0 [10.133.0.125/23] from ovn-kubernetes 44m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Container created 44m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Container started 44m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Container started 44m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Container created 44m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Add eth0 [10.134.0.108/23] from ovn-kubernetes 44m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Container started 44m Warning Unhealthy pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Startup probe failed: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused 44m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Add eth0 [10.134.0.109/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Startup probe error: Get "http://10.132.0.180:15021/healthz/ready": dial tcp 10.132.0.180:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Startup probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Startup probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 44m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Startup probe failed: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused 44m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Startup probe error: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Startup probe failed: Get "http://10.134.0.109:15021/healthz/ready": dial tcp 10.134.0.109:15021: connect: connection refused 44m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Startup probe error: Get "http://10.134.0.108:15021/healthz/ready": dial tcp 10.134.0.108:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-0390e1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-0390e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Killing pod/payload-pre-processing-e2e-api-b-004180-845f96955-lw94h Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-api-b-004180-548d486474-4m87j Stopping container payload-processing 44m Normal Killing pod/e2e-api-a-a19582-openshift-default-7df57db669-fzn8r Stopping container istio-proxy 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-702002-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-702002-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x 44m Normal ScalingReplicaSet deployment/e2e-mt-21dd16de-openshift-default Scaled up replica set e2e-mt-21dd16de-openshift-default-5db99d957 from 4 to 7 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-21dd16de-openshift-default New size: 7; reason: All metrics below target 44m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-5jthv 44m Normal SuccessfulCreate replicaset/e2e-mt-21dd16de-openshift-default-5db99d957 Created pod: e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr 44m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Add eth0 [10.132.0.195/23] from ovn-kubernetes 44m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Container started 44m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Container created 44m Normal AddedLabel gateway/e2e-inf-a-d0a5de Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-d0a5de 44m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Add eth0 [10.133.0.126/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-inf-a-d0a5de-openshift-default Scaled up replica set e2e-inf-a-d0a5de-openshift-default-79f4bc7977 from 0 to 1 44m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Container created 44m Normal Pulled pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Container created 44m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Container started 44m Normal AddedInterface pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Add eth0 [10.132.0.196/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-inf-a-d0a5de-openshift-default-79f4bc7977 Created pod: e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv 44m Normal AddedInterface pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Add eth0 [10.132.0.197/23] from ovn-kubernetes 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-004180-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-004180-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal Started pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Container started 44m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Unable to retrieve some image pull secrets (e2e-mt-21dd16de-openshift-default-dockercfg-8mdj7); attempting to pull the image may not succeed. 44m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Startup probe failed: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused 44m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Startup probe error: Get "http://10.133.0.126:15021/healthz/ready": dial tcp 10.133.0.126:15021: connect: connection refused... 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-gqqjr Stopping container istio-proxy 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-xm2fp Stopping container istio-proxy 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-tpbcg Stopping container istio-proxy 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-dnvq4 Stopping container istio-proxy 44m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Startup probe error: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Startup probe error: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused... 44m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Unable to retrieve some image pull secrets (e2e-mt-21dd16de-openshift-default-dockercfg-8mdj7); attempting to pull the image may not succeed. 44m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Startup probe failed: Get "http://10.132.0.195:15021/healthz/ready": dial tcp 10.132.0.195:15021: connect: connection refused 44m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Unable to retrieve some image pull secrets (e2e-mt-21dd16de-openshift-default-dockercfg-8mdj7); attempting to pull the image may not succeed. 44m Warning Unhealthy pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Startup probe failed: Get "http://10.132.0.196:15021/healthz/ready": dial tcp 10.132.0.196:15021: connect: connection refused 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-xdrkr Stopping container istio-proxy 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-5jthv Stopping container istio-proxy 44m Normal Killing pod/e2e-mt-21dd16de-openshift-default-5db99d957-8mc6x Stopping container istio-proxy 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-d0a5de Scaled up replica set payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-d0a5de Scaled up replica set payload-processing-e2e-inf-a-d0a5de-5989495d4b from 0 to 1 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Add eth0 [10.134.0.111/23] from ovn-kubernetes 44m Normal Pulled pod/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv Add eth0 [10.134.0.112/23] from ovn-kubernetes 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-d0a5de-5989495d4b Error creating: pods "payload-processing-e2e-inf-a-d0a5de-5989495d4b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d0a5de: serviceaccount "payload-processing-e2e-inf-a-d0a5de" not found 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Container created 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-d0a5de-5989495d4b Created pod: payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c Created pod: payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c Error creating: pods "payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-d0a5de: serviceaccount "payload-processing-e2e-inf-a-d0a5de" not found 44m Normal Started pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Container started 44m Normal Started pod/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv Container started 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 44m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-q95p5 Stopping container istio-proxy 44m Normal Created pod/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv Container created 44m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled down replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 6 to 5 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 5; reason: All metrics below target 44m Normal Killing pod/payload-pre-processing-e2e-api-b-0390e1-6b87bf79c5-4plts Stopping container payload-pre-processing 44m Normal Killing pod/payload-processing-e2e-api-b-0390e1-7b5b74b5c6-dvxpr Stopping container payload-processing 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-0390e1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-0390e1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn 44m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-bf7xn Stopping container istio-proxy 44m Normal Killing pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Stopping container istio-proxy 44m Normal AddedLabel gateway/e2e-inf-a-17957f Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-17957f 44m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled up replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 0 to 1 44m Normal Pulled pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Container started 44m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 44m Normal Created pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Container created 44m Normal Pulled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Container created 44m Normal SuccessfulCreate replicaset/e2e-inf-b-3577a0-openshift-default-84498fd47 Created pod: e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 44m Normal SuccessfulCreate replicaset/e2e-inf-b-3577a0-openshift-default-84498fd47 Created pod: e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 44m Warning Unhealthy pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Readiness probe failed: HTTP probe failed with statuscode: 503 44m Normal AddedInterface pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Add eth0 [10.134.0.113/23] from ovn-kubernetes 44m Normal AddedInterface pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Add eth0 [10.132.0.199/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-inf-b-3577a0-openshift-default-84498fd47 Created pod: e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc 44m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Add eth0 [10.133.0.128/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-api-b-004180-openshift-default-7dfb78476f-25d2k Readiness probe error: HTTP probe failed with statuscode: 503... 44m Warning ProbeError pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Readiness probe error: Get "http://10.132.0.186:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 44m Warning Unhealthy pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Readiness probe failed: Get "http://10.132.0.186:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 44m Normal AddedInterface pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Add eth0 [10.133.0.129/23] from ovn-kubernetes 44m Normal Started pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Container started 44m Normal ScalingReplicaSet deployment/e2e-inf-b-3577a0-openshift-default Scaled up replica set e2e-inf-b-3577a0-openshift-default-84498fd47 from 1 to 4 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3577a0-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 44m Normal Pulled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Container created 44m Normal Started pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Container started 44m Normal Pulled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Container started 44m Normal Created pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Container created 44m Normal Killing pod/payload-processing-e2e-api-a-bb882c-8486dc96fb-llqsj Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-api-a-bb882c-c9cf8d987-ljp4g Stopping container payload-pre-processing 44m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-17957f-5686759fdb Created pod: payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm 44m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-de0cba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 44m Normal ScalingReplicaSet deployment/e2e-inf-a-edd32a-openshift-default Scaled up replica set e2e-inf-a-edd32a-openshift-default-7cf5f49d65 from 1 to 4 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-17957f-86785bdb55 Error creating: pods "payload-pre-processing-e2e-inf-a-17957f-86785bdb55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-17957f: serviceaccount "payload-processing-e2e-inf-a-17957f" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-17957f Scaled up replica set payload-processing-e2e-inf-a-17957f-5686759fdb from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-17957f-86785bdb55 Created pod: payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-17957f Scaled up replica set payload-pre-processing-e2e-inf-a-17957f-86785bdb55 from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-17957f-5686759fdb Error creating: pods "payload-processing-e2e-inf-a-17957f-5686759fdb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-17957f: serviceaccount "payload-processing-e2e-inf-a-17957f" not found 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-de0cba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Add eth0 [10.134.0.115/23] from ovn-kubernetes 44m Normal Started pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Container started 44m Normal Created pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Container created 44m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Container started 44m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Container created 44m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Add eth0 [10.132.0.200/23] from ovn-kubernetes 44m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Container created 44m Normal Created pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Container created 44m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Container started 44m Normal Started pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Container started 44m Normal Created pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Container created 44m Normal Pulled pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Container created 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Add eth0 [10.133.0.130/23] from ovn-kubernetes 44m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Container started 44m Normal Pulled pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Add eth0 [10.134.0.114/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Container started 44m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 44m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp 44m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled down replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 6 to 5 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 4; reason: All metrics below target 44m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Add eth0 [10.133.0.131/23] from ovn-kubernetes 44m Warning ProbeError pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 44m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph 44m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Container started 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 5; reason: All metrics below target 44m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Container created 44m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Container started 44m Warning ProbeError pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Startup probe error: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Startup probe failed: Get "http://10.134.0.113:15021/healthz/ready": dial tcp 10.134.0.113:15021: connect: connection refused 44m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Stopping container istio-proxy 43m Warning Unhealthy pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Startup probe error: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cbpph Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Startup probe failed: Get "http://10.132.0.199:15021/healthz/ready": dial tcp 10.132.0.199:15021: connect: connection refused 43m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-858wp Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Startup probe failed: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Startup probe error: Get "http://10.134.0.115:15021/healthz/ready": dial tcp 10.134.0.115:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Startup probe failed: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Startup probe error: Get "http://10.133.0.131:15021/healthz/ready": dial tcp 10.133.0.131:15021: connect: connection refused... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedLabel gateway/e2e-inf-b-9d8b5d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-9d8b5d 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal SuccessfulCreate replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Created pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm 43m Normal ScalingReplicaSet deployment/e2e-inf-b-9d8b5d-openshift-default Scaled up replica set e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc from 0 to 1 43m Warning Unhealthy pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Startup probe failed: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused 43m Warning ProbeError pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Startup probe error: Get "http://10.132.0.186:15021/healthz/ready": dial tcp 10.132.0.186:15021: connect: connection refused... 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3577a0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3577a0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning Unhealthy pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Startup probe failed: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Startup probe error: Get "http://10.132.0.183:15021/healthz/ready": dial tcp 10.132.0.183:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Startup probe error: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Startup probe failed: Get "http://10.132.0.179:15021/healthz/ready": dial tcp 10.132.0.179:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Startup probe failed: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Startup probe error: Get "http://10.132.0.200:15021/healthz/ready": dial tcp 10.132.0.200:15021: connect: connection refused... 43m Normal AddedInterface pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Add eth0 [10.132.0.201/23] from ovn-kubernetes 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-9d8b5d Scaled up replica set payload-processing-e2e-inf-b-9d8b5d-76d68746b6 from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-9d8b5d-76d68746b6 Error creating: pods "payload-processing-e2e-inf-b-9d8b5d-76d68746b6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9d8b5d: serviceaccount "payload-processing-e2e-inf-b-9d8b5d" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-9d8b5d Scaled up replica set payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99 Created pod: payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99 Error creating: pods "payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-9d8b5d: serviceaccount "payload-processing-e2e-inf-b-9d8b5d" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-9d8b5d-76d68746b6 Created pod: payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh 43m Normal Created pod/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh Container created 43m Normal Created pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Add eth0 [10.133.0.132/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh Add eth0 [10.134.0.117/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Container started 43m Normal Started pod/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh Container started 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-bb882c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-bb882c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedLabel gateway/e2e-api-a-0f9b7e Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-0f9b7e 43m Normal AddedInterface pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Add eth0 [10.132.0.202/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/e2e-api-a-0f9b7e-openshift-default Scaled up replica set e2e-api-a-0f9b7e-openshift-default-b776cc9f from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-api-a-0f9b7e-openshift-default-b776cc9f Created pod: e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv 43m Normal Killing pod/e2e-api-b-0390e1-openshift-default-5499d6bdcf-v25t7 Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-inf-a-f2760d-openshift-default Scaled up replica set e2e-inf-a-f2760d-openshift-default-845f578cdf from 0 to 1 43m Normal AddedLabel gateway/e2e-inf-a-f2760d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f2760d 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f2760d-openshift-default-845f578cdf Created pod: e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3577a0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3577a0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedInterface pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Add eth0 [10.132.0.203/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f Created pod: payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-0f9b7e Scaled up replica set payload-processing-e2e-api-a-0f9b7e-7456c8d5cb from 0 to 1 43m Normal Pulled pod/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg Add eth0 [10.134.0.118/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb Error creating: pods "payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0f9b7e: serviceaccount "payload-processing-e2e-api-a-0f9b7e" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f Error creating: pods "payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-0f9b7e: serviceaccount "payload-processing-e2e-api-a-0f9b7e" not found 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-0f9b7e Scaled up replica set payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb Created pod: payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg 43m Normal Started pod/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx Container started 43m Normal Created pod/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg Container created 43m Normal Pulled pod/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx Container created 43m Normal Started pod/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx Add eth0 [10.133.0.133/23] from ovn-kubernetes 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f2760d-577bdfbb74 Created pod: payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-de0cba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-de0cba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f2760d Scaled up replica set payload-pre-processing-e2e-inf-a-f2760d-d96bc568c from 0 to 1 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f2760d-577bdfbb74 Error creating: pods "payload-processing-e2e-inf-a-f2760d-577bdfbb74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f2760d: serviceaccount "payload-processing-e2e-inf-a-f2760d" not found 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c Created pod: payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f2760d Scaled up replica set payload-processing-e2e-inf-a-f2760d-577bdfbb74 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c Error creating: pods "payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f2760d: serviceaccount "payload-processing-e2e-inf-a-f2760d" not found 43m Normal Pulled pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Add eth0 [10.134.0.119/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Container started 43m Normal Created pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Add eth0 [10.133.0.134/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Container created 43m Normal Started pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Container started 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-bb882c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-bb882c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz 43m Normal AddedLabel gateway/e2e-inf-b-830149 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-830149 43m Normal ScalingReplicaSet deployment/e2e-inf-b-830149-openshift-default Scaled up replica set e2e-inf-b-830149-openshift-default-64cc8cfc59 from 0 to 1 43m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Add eth0 [10.133.0.136/23] from ovn-kubernetes 43m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Readiness probe error: Get "http://10.132.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Readiness probe failed: Get "http://10.132.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Killing pod/e2e-api-a-bb882c-openshift-default-64f5fbfd9f-8lkjz Stopping container istio-proxy 43m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht 43m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb 43m Normal ScalingReplicaSet deployment/e2e-inf-a-edd32a-openshift-default Scaled up replica set e2e-inf-a-edd32a-openshift-default-7cf5f49d65 from 4 to 7 43m Normal SuccessfulCreate replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Created pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default New size: 7; reason: All metrics below target 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-830149-58d9646cff Error creating: pods "payload-pre-processing-e2e-inf-b-830149-58d9646cff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-830149: serviceaccount "payload-processing-e2e-inf-b-830149" not found 43m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Add eth0 [10.132.0.205/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Container started 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-830149 Scaled up replica set payload-processing-e2e-inf-b-830149-7f864c9c64 from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-830149 Scaled up replica set payload-pre-processing-e2e-inf-b-830149-58d9646cff from 0 to 1 43m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Container created 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-830149-58d9646cff Created pod: payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-830149-7f864c9c64 Created pod: payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m 43m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Container created 43m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Add eth0 [10.133.0.137/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Add eth0 [10.134.0.120/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-830149-7f864c9c64 Error creating: pods "payload-processing-e2e-inf-b-830149-7f864c9c64-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-830149: serviceaccount "payload-processing-e2e-inf-b-830149" not found 43m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Container started 43m Normal Started pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Container started 43m Normal Created pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Container created 43m Normal Pulled pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Add eth0 [10.134.0.122/23] from ovn-kubernetes 43m Normal Started pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Add eth0 [10.134.0.121/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Container started 43m Normal Created pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Container created 43m Normal Created pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Container created 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Startup probe failed: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Startup probe error: Get "http://10.134.0.120:15021/healthz/ready": dial tcp 10.134.0.120:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Startup probe failed: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Startup probe error: Get "http://10.132.0.205:15021/healthz/ready": dial tcp 10.132.0.205:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Startup probe failed: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused 43m Normal Killing pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Stopping container payload-pre-processing 43m Warning ProbeError pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Startup probe error: Get "http://10.133.0.137:15021/healthz/ready": dial tcp 10.133.0.137:15021: connect: connection refused... 43m Normal Killing pod/payload-processing-e2e-api-b-18657f-865b44ff74-n8hw6 Stopping container payload-processing 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz 43m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp 43m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr 43m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Add eth0 [10.134.0.124/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Container started 43m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled up replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 1 to 4 43m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Container created 43m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Container created 43m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Container started 43m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Container created 43m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Container created 43m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Add eth0 [10.132.0.207/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Container started 43m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Container started 43m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Add eth0 [10.132.0.206/23] from ovn-kubernetes 43m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Startup probe failed: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused 43m Warning Unhealthy pod/payload-pre-processing-e2e-api-b-18657f-7f9f69fb5f-gwxwd Readiness probe failed: timeout: failed to connect service "10.134.0.107:50052" within 1s: context deadline exceeded 43m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Startup probe error: Get "http://10.134.0.124:15021/healthz/ready": dial tcp 10.134.0.124:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Startup probe failed: Get "http://10.132.0.206:15021/healthz/ready": dial tcp 10.132.0.206:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Startup probe error: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Startup probe failed: Get "http://10.132.0.207:15021/healthz/ready": dial tcp 10.132.0.207:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Startup probe error: Get "http://10.132.0.206:15021/healthz/ready": dial tcp 10.132.0.206:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Readiness probe failed: Get "http://10.132.0.191:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Readiness probe error: Get "http://10.132.0.191:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Pulled pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Container created 42m Normal Started pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Container started 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal AddedLabel gateway/e2e-api-b-e2a91d Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-e2a91d 42m Normal SuccessfulCreate replicaset/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9 Created pod: e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc 42m Normal ScalingReplicaSet deployment/e2e-api-b-e2a91d-openshift-default Scaled up replica set e2e-api-b-e2a91d-openshift-default-66f9fb6fc9 from 0 to 1 42m Normal AddedInterface pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Add eth0 [10.132.0.209/23] from ovn-kubernetes 42m Warning ProbeError pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Startup probe error: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Startup probe failed: Get "http://10.132.0.191:15021/healthz/ready": dial tcp 10.132.0.191:15021: connect: connection refused 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-18657f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-18657f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-e2a91d Scaled up replica set payload-processing-e2e-api-b-e2a91d-788c9d6d68 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c Created pod: payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-e2a91d Scaled up replica set payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c from 0 to 1 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c Error creating: pods "payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-e2a91d: serviceaccount "payload-processing-e2e-api-b-e2a91d" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-e2a91d-788c9d6d68 Created pod: payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc 42m Warning FailedCreate replicaset/payload-processing-e2e-api-b-e2a91d-788c9d6d68 Error creating: pods "payload-processing-e2e-api-b-e2a91d-788c9d6d68-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-e2a91d: serviceaccount "payload-processing-e2e-api-b-e2a91d" not found 42m Normal Pulled pod/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv Add eth0 [10.133.0.140/23] from ovn-kubernetes 42m Normal Created pod/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc Container created 42m Normal Started pod/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv Container started 42m Normal Started pod/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc Container started 42m Normal Created pod/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv Container created 42m Normal Pulled pod/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc Add eth0 [10.133.0.141/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-inf-b-8483c2-openshift-default-86687957bf Created pod: e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm 42m Normal AddedLabel gateway/e2e-inf-b-8483c2 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-8483c2 42m Normal ScalingReplicaSet deployment/e2e-inf-b-8483c2-openshift-default Scaled up replica set e2e-inf-b-8483c2-openshift-default-86687957bf from 0 to 1 42m Normal AddedInterface pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Add eth0 [10.132.0.210/23] from ovn-kubernetes 42m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Container created 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default New size: 7; reason: 42m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Add eth0 [10.133.0.142/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-8483c2 Scaled up replica set payload-pre-processing-e2e-inf-b-8483c2-79469b67b7 from 0 to 1 42m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled up replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 4 to 7 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7 Error creating: pods "payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8483c2: serviceaccount "payload-processing-e2e-inf-b-8483c2" not found 42m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-8483c2 Scaled up replica set payload-processing-e2e-inf-b-8483c2-57fd7d7846 from 0 to 1 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-8483c2-57fd7d7846 Created pod: payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7 Created pod: payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw 42m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Container created 42m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Container started 42m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Add eth0 [10.133.0.143/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz 42m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Add eth0 [10.134.0.125/23] from ovn-kubernetes 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-8483c2-57fd7d7846 Error creating: pods "payload-processing-e2e-inf-b-8483c2-57fd7d7846-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-8483c2: serviceaccount "payload-processing-e2e-inf-b-8483c2" not found 42m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw 42m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt 42m Normal Created pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Container created 42m Normal Created pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Container created 42m Normal AddedInterface pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Add eth0 [10.134.0.127/23] from ovn-kubernetes 42m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Container started 42m Normal Started pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Container started 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Add eth0 [10.134.0.126/23] from ovn-kubernetes 42m Normal Pulled pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Container started 42m Normal Started pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Container started 42m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Container created 42m Normal Pulled pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-18657f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-18657f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 42m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Startup probe error: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Startup probe failed: Get "http://10.134.0.125:15021/healthz/ready": dial tcp 10.134.0.125:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Startup probe failed: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused 42m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Startup probe failed: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Startup probe error: Get "http://10.133.0.143:15021/healthz/ready": dial tcp 10.133.0.143:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Startup probe error: Get "http://10.133.0.142:15021/healthz/ready": dial tcp 10.133.0.142:15021: connect: connection refused... 42m Normal Killing pod/e2e-api-b-18657f-openshift-default-54fcbd9989-5w5bz Stopping container istio-proxy 42m Normal AddedLabel gateway/e2e-inf-a-487639 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-487639 42m Normal SuccessfulCreate replicaset/e2e-inf-a-487639-openshift-default-5d5fc9449c Created pod: e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 42m Normal ScalingReplicaSet deployment/e2e-inf-a-487639-openshift-default Scaled up replica set e2e-inf-a-487639-openshift-default-5d5fc9449c from 0 to 1 42m Normal AddedInterface pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Add eth0 [10.132.0.214/23] from ovn-kubernetes 42m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-487639 Scaled up replica set payload-pre-processing-e2e-inf-a-487639-6bc59bb99f from 0 to 1 42m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-487639-cdbf7c87d Error creating: pods "payload-processing-e2e-inf-a-487639-cdbf7c87d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-487639: serviceaccount "payload-processing-e2e-inf-a-487639" not found 42m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-487639 Scaled up replica set payload-processing-e2e-inf-a-487639-cdbf7c87d from 0 to 1 42m Normal Pulled pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Add eth0 [10.134.0.129/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f Created pod: payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv 42m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f Error creating: pods "payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-487639: serviceaccount "payload-processing-e2e-inf-a-487639" not found 42m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-487639-cdbf7c87d Created pod: payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 42m Normal Created pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Container created 42m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Add eth0 [10.133.0.144/23] from ovn-kubernetes 42m Normal Started pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Container started 42m Normal Created pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Container created 42m Normal Started pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Container started 42m Normal Pulled pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Add eth0 [10.133.0.145/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Created pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd 42m Normal SuccessfulCreate replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Created pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 42m Normal Pulled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Started pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Container started 42m Normal Created pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Container created 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 42m Normal Pulled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal AddedInterface pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Add eth0 [10.134.0.131/23] from ovn-kubernetes 42m Normal SuccessfulCreate replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Created pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng 42m Normal ScalingReplicaSet deployment/e2e-inf-b-9d8b5d-openshift-default Scaled up replica set e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc from 1 to 4 42m Normal Started pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Container started 42m Warning Unhealthy pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Readiness probe failed: Get "http://10.132.0.202:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Readiness probe error: Get "http://10.132.0.202:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal Pulled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Container created 42m Normal Created pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Container created 42m Normal AddedInterface pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Add eth0 [10.132.0.216/23] from ovn-kubernetes 42m Normal Started pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Container started 42m Warning Unhealthy pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Readiness probe failed: Get "http://10.132.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning Unhealthy pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Readiness probe failed: Get "http://10.132.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Readiness probe error: Get "http://10.132.0.203:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning ProbeError pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Readiness probe error: Get "http://10.132.0.197:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Readiness probe failed: Get "http://10.132.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Startup probe failed: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Startup probe error: Get "http://10.134.0.131:15021/healthz/ready": dial tcp 10.134.0.131:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Readiness probe error: Get "http://10.132.0.201:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal SuccessfulCreate replicaset/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67 Created pod: e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs 42m Normal ScalingReplicaSet deployment/e2e-inf-a-dfdb11-openshift-default Scaled up replica set e2e-inf-a-dfdb11-openshift-default-75fbd6bd67 from 0 to 1 42m Normal AddedLabel gateway/e2e-inf-a-dfdb11 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-dfdb11 42m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused... 41m Normal Started pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Container started 41m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Startup probe error: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Startup probe failed: Get "http://10.132.0.216:15021/healthz/ready": dial tcp 10.132.0.216:15021: connect: connection refused 41m Normal Created pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Container created 41m Normal Pulled pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Add eth0 [10.132.0.217/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Container created 41m Normal Started pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Container started 41m Normal Created pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Container created 41m Normal Started pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Container started 41m Normal Created pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Container created 41m Normal Started pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-487639-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-487639-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc Error creating: pods "payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-dfdb11: serviceaccount "payload-processing-e2e-inf-a-dfdb11" not found 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-dfdb11 Scaled up replica set payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc from 0 to 1 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d0a5de-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-dfdb11 Scaled up replica set payload-processing-e2e-inf-a-dfdb11-fd9d47844 from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-dfdb11-fd9d47844 Created pod: payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-dfdb11-fd9d47844 Error creating: pods "payload-processing-e2e-inf-a-dfdb11-fd9d47844-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-dfdb11: serviceaccount "payload-processing-e2e-inf-a-dfdb11" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc Created pod: payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d0a5de-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Created pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Container created 41m Normal Started pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Container started 41m Normal Started pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Container started 41m Normal Pulled pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Add eth0 [10.134.0.132/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Add eth0 [10.134.0.133/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Container created 41m Warning ProbeError pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Startup probe error: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Startup probe failed: Get "http://10.132.0.197:15021/healthz/ready": dial tcp 10.132.0.197:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Startup probe error: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Startup probe failed: Get "http://10.132.0.201:15021/healthz/ready": dial tcp 10.132.0.201:15021: connect: connection refused 41m Warning ProbeError pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Startup probe error: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Startup probe failed: Get "http://10.132.0.202:15021/healthz/ready": dial tcp 10.132.0.202:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Startup probe error: Get "http://10.132.0.203:15021/healthz/ready": dial tcp 10.132.0.203:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Startup probe failed: Get "http://10.132.0.203:15021/healthz/ready": dial tcp 10.132.0.203:15021: connect: connection refused 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0f9b7e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0f9b7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal Killing pod/payload-pre-processing-e2e-api-a-0f9b7e-6f4456d5f-fchfx Stopping container payload-pre-processing 41m Normal Killing pod/payload-processing-e2e-api-a-0f9b7e-7456c8d5cb-dfzvg Stopping container payload-processing 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-d0a5de-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-d0a5de-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm 41m Normal ScalingReplicaSet deployment/e2e-inf-b-3e300c-openshift-default Scaled up replica set e2e-inf-b-3e300c-openshift-default-679f5f7777 from 0 to 1 41m Normal AddedLabel gateway/e2e-inf-b-3e300c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-3e300c 41m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Add eth0 [10.134.0.134/23] from ovn-kubernetes 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-dfdb11-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-dfdb11-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-3e300c-74b58496db Error creating: pods "payload-processing-e2e-inf-b-3e300c-74b58496db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3e300c: serviceaccount "payload-processing-e2e-inf-b-3e300c" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-3e300c-74b58496db Created pod: payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99 Error creating: pods "payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-3e300c: serviceaccount "payload-processing-e2e-inf-b-3e300c" not found 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99 Created pod: payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-3e300c Scaled up replica set payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99 from 0 to 1 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-3e300c Scaled up replica set payload-processing-e2e-inf-b-3e300c-74b58496db from 0 to 1 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Add eth0 [10.133.0.148/23] from ovn-kubernetes 41m Normal Started pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Container started 41m Normal Created pod/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh Container created 41m Normal Created pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Container created 41m Normal Pulled pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Started pod/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh Container started 41m Normal AddedInterface pod/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh Add eth0 [10.134.0.135/23] from ovn-kubernetes 41m Normal Pulled pod/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f2760d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f2760d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Created pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m 41m Normal ScalingReplicaSet deployment/e2e-inf-b-9d8b5d-openshift-default Scaled up replica set e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc from 4 to 5 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default New size: 5; reason: All metrics below target 41m Normal Started pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Container started 41m Normal AddedInterface pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Add eth0 [10.134.0.136/23] from ovn-kubernetes 41m Normal Created pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Container created 41m Normal Pulled pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Readiness probe error: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Readiness probe error: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Readiness probe failed: Get "http://10.132.0.54:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Readiness probe failed: Get "http://10.133.0.43:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 41m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Startup probe error: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Startup probe failed: Get "http://10.134.0.136:15021/healthz/ready": dial tcp 10.134.0.136:15021: connect: connection refused 41m Warning Unhealthy pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Readiness probe failed: timeout: failed to connect service "10.132.0.59:50052" within 1s: context deadline exceeded 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-0f9b7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-0f9b7e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-487639-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal SuccessfulCreate replicaset/e2e-inf-a-487639-openshift-default-5d5fc9449c Created pod: e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg 41m Normal ScalingReplicaSet deployment/e2e-inf-a-487639-openshift-default Scaled up replica set e2e-inf-a-487639-openshift-default-5d5fc9449c from 1 to 4 41m Normal SuccessfulCreate replicaset/e2e-inf-a-487639-openshift-default-5d5fc9449c Created pod: e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd 41m Normal SuccessfulCreate replicaset/e2e-inf-a-487639-openshift-default-5d5fc9449c Created pod: e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw 41m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f 41m Normal Started pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Container started 41m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq 41m Normal SuccessfulCreate replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Created pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled up replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 7 to 9 41m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx 41m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p 41m Normal AddedInterface pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Add eth0 [10.132.0.221/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Container created 41m Normal Started pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Container started 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-830149-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 41m Normal ScalingReplicaSet deployment/e2e-inf-b-830149-openshift-default Scaled up replica set e2e-inf-b-830149-openshift-default-64cc8cfc59 from 1 to 4 41m Normal AddedInterface pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Add eth0 [10.132.0.222/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Container created 41m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Container started 41m Normal Started pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Container started 41m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Container created 41m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Add eth0 [10.134.0.137/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Container created 41m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Add eth0 [10.132.0.224/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Add eth0 [10.132.0.225/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Add eth0 [10.132.0.223/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Container started 41m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Container started 41m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Container created 41m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Container created 41m Normal Created pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Container created 41m Normal Started pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Container started 41m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 41m Normal AddedLabel gateway/e2e-inf-b-070e3d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-070e3d 41m Normal SuccessfulCreate replicaset/e2e-inf-b-070e3d-openshift-default-68c88bd895 Created pod: e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 41m Normal ScalingReplicaSet deployment/e2e-inf-b-070e3d-openshift-default Scaled up replica set e2e-inf-b-070e3d-openshift-default-68c88bd895 from 0 to 1 41m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Readiness probe failed: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Readiness probe error: Get "http://10.132.0.209:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Startup probe failed: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Startup probe failed: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Readiness probe error: Get "http://10.132.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Readiness probe failed: Get "http://10.132.0.210:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Startup probe error: Get "http://10.132.0.222:15021/healthz/ready": dial tcp 10.132.0.222:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Startup probe error: Get "http://10.132.0.221:15021/healthz/ready": dial tcp 10.132.0.221:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Startup probe failed: Get "http://10.132.0.225:15021/healthz/ready": dial tcp 10.132.0.225:15021: connect: connection refused 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-830149-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-830149-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Startup probe failed: Get "http://10.132.0.224:15021/healthz/ready": dial tcp 10.132.0.224:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Startup probe error: Get "http://10.132.0.225:15021/healthz/ready": dial tcp 10.132.0.225:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Startup probe failed: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Startup probe error: Get "http://10.132.0.224:15021/healthz/ready": dial tcp 10.132.0.224:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Startup probe error: Get "http://10.132.0.223:15021/healthz/ready": dial tcp 10.132.0.223:15021: connect: connection refused... 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-api-b-e2a91d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 40m Normal ScalingReplicaSet deployment/e2e-api-b-e2a91d-openshift-default Scaled up replica set e2e-api-b-e2a91d-openshift-default-66f9fb6fc9 from 1 to 4 40m Normal SuccessfulCreate replicaset/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9 Created pod: e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-gj2hp 40m Normal SuccessfulCreate replicaset/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9 Created pod: e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-m6nc9 40m Normal SuccessfulCreate replicaset/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9 Created pod: e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-twqtc 40m Warning Unhealthy pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Readiness probe failed: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Readiness probe error: Get "http://10.132.0.214:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Warning Unhealthy pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Readiness probe failed: Get "http://10.132.0.209:15021/healthz/ready": read tcp 10.132.0.2:55502->10.132.0.209:15021: read: connection reset by peer 40m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Add eth0 [10.134.0.138/23] from ovn-kubernetes 40m Warning ProbeError pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Readiness probe error: Get "http://10.132.0.209:15021/healthz/ready": read tcp 10.132.0.2:55502->10.132.0.209:15021: read: connection reset by peer... 40m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Container created 40m Normal Created pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Container created 40m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Stopping container istio-proxy 40m Normal Created pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Container created 40m Normal Started pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Container started 40m Normal Pulled pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Container started 40m Normal Pulled pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Container started 40m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 4; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default Scaled down replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 5 to 4 40m Normal Created pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Container created 40m Normal Pulled pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Container started 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-487639-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Startup probe failed: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-487639-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Startup probe error: Get "http://10.134.0.138:15021/healthz/ready": dial tcp 10.134.0.138:15021: connect: connection refused... 40m Normal Killing pod/e2e-api-a-0f9b7e-openshift-default-b776cc9f-z7gbv Stopping container istio-proxy 40m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Readiness probe failed: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gczng Readiness probe error: Get "http://10.132.0.69:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-830149-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-830149-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-e2a91d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Startup probe failed: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-e2a91d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Startup probe error: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Startup probe failed: Get "http://10.132.0.210:15021/healthz/ready": dial tcp 10.132.0.210:15021: connect: connection refused 40m Warning Unhealthy pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Startup probe failed: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Startup probe error: Get "http://10.132.0.214:15021/healthz/ready": dial tcp 10.132.0.214:15021: connect: connection refused... 40m Warning ProbeError pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Startup probe error: Get "http://10.132.0.209:15021/healthz/ready": dial tcp 10.132.0.209:15021: connect: connection refused... 40m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Add eth0 [10.132.0.227/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Add eth0 [10.132.0.228/23] from ovn-kubernetes 40m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Container created 40m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Container started 40m Normal Killing pod/payload-pre-processing-e2e-api-b-e2a91d-544d8bc56c-m9tgv Stopping container payload-pre-processing 40m Normal Killing pod/payload-processing-e2e-api-b-e2a91d-788c9d6d68-fb2lc Stopping container payload-processing 40m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-070e3d-6b97d99dd Error creating: pods "payload-processing-e2e-inf-b-070e3d-6b97d99dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-070e3d: serviceaccount "payload-processing-e2e-inf-b-070e3d" not found 40m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c Created pod: payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 40m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c Error creating: pods "payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-070e3d: serviceaccount "payload-processing-e2e-inf-b-070e3d" not found 40m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-070e3d-6b97d99dd Created pod: payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf 40m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-070e3d Scaled up replica set payload-processing-e2e-inf-b-070e3d-6b97d99dd from 0 to 1 40m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-070e3d Scaled up replica set payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c from 0 to 1 40m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Add eth0 [10.133.0.149/23] from ovn-kubernetes 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8483c2-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Created pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Container created 40m Normal Started pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Container started 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8483c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Normal Pulled pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf Add eth0 [10.132.0.229/23] from ovn-kubernetes 40m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Startup probe failed: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused 40m Normal Started pod/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf Container started 40m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Startup probe error: Get "http://10.132.0.227:15021/healthz/ready": dial tcp 10.132.0.227:15021: connect: connection refused... 40m Normal Created pod/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf Container created 40m Normal Pulled pod/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 40m Normal ScalingReplicaSet deployment/e2e-inf-b-830149-openshift-default Scaled up replica set e2e-inf-b-830149-openshift-default-64cc8cfc59 from 4 to 8 40m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s 40m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb 40m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg 40m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt 40m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default Scaled down replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 7 to 6 40m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-8dx9d 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 6; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled down replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 7 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 40m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-8spl5 Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default Scaled down replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 7 to 6 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-spwtx 40m Normal Pulled pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Container started 40m Normal Created pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Container created 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-8483c2-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-8483c2-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Add eth0 [10.134.0.140/23] from ovn-kubernetes 40m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Container started 40m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Container created 40m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Add eth0 [10.132.0.230/23] from ovn-kubernetes 40m Normal ScalingReplicaSet deployment/e2e-inf-b-830149-openshift-default Scaled up replica set e2e-inf-b-830149-openshift-default-64cc8cfc59 from 8 to 9 40m Normal SuccessfulCreate replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Created pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww 40m Normal ScalingReplicaSet deployment/e2e-inf-a-f2760d-openshift-default Scaled up replica set e2e-inf-a-f2760d-openshift-default-845f578cdf from 1 to 2 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-830149-openshift-default New size: 9; reason: All metrics below target 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f2760d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 40m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Container created 40m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal SuccessfulCreate replicaset/e2e-inf-a-f2760d-openshift-default-845f578cdf Created pod: e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c 40m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Container started 40m Warning Unhealthy pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Startup probe failed: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Startup probe error: Get "http://10.132.0.217:15021/healthz/ready": dial tcp 10.132.0.217:15021: connect: connection refused... 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-e2a91d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-e2a91d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Startup probe failed: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Startup probe error: Get "http://10.134.0.140:15021/healthz/ready": dial tcp 10.134.0.140:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Startup probe failed: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Startup probe error: Get "http://10.132.0.230:15021/healthz/ready": dial tcp 10.132.0.230:15021: connect: connection refused... 39m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Container started 39m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Container created 39m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Add eth0 [10.134.0.141/23] from ovn-kubernetes 39m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Startup probe error: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Startup probe failed: Get "http://10.134.0.141:15021/healthz/ready": dial tcp 10.134.0.141:15021: connect: connection refused 39m Normal Killing pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Stopping container istio-proxy 39m Normal ScalingReplicaSet deployment/e2e-inf-a-e7e2df-openshift-default Scaled up replica set e2e-inf-a-e7e2df-openshift-default-584f54d89c from 0 to 1 39m Normal SuccessfulCreate replicaset/e2e-inf-a-e7e2df-openshift-default-584f54d89c Created pod: e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z 39m Normal AddedLabel gateway/e2e-inf-a-e7e2df Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-e7e2df 39m Warning ProbeError pod/e2e-api-b-e2a91d-openshift-default-66f9fb6fc9-n9swc Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Add eth0 [10.132.0.232/23] from ovn-kubernetes 39m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Container started 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-dfdb11-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-dfdb11-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Container created 39m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Startup probe failed: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Startup probe error: Get "http://10.132.0.232:15021/healthz/ready": dial tcp 10.132.0.232:15021: connect: connection refused... 39m Normal ScalingReplicaSet deployment/e2e-inf-b-3577a0-openshift-default Scaled down replica set e2e-inf-b-3577a0-openshift-default-84498fd47 from 4 to 3 39m Normal SuccessfulDelete replicaset/e2e-inf-b-3577a0-openshift-default-84498fd47 Deleted pod: e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3577a0-openshift-default New size: 3; reason: All metrics below target 39m Normal Killing pod/e2e-inf-b-3577a0-openshift-default-84498fd47-2r6p8 Stopping container istio-proxy 38m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Readiness probe failed: timeout: health rpc did not complete within 1s 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Readiness probe failed: timeout: failed to connect service "10.134.0.122:50052" within 1s: context deadline exceeded 38m Normal Created pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Container created 38m Normal Started pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Container started 38m Normal Pulled pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3e300c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 38m Normal ScalingReplicaSet deployment/e2e-inf-b-3e300c-openshift-default Scaled up replica set e2e-inf-b-3e300c-openshift-default-679f5f7777 from 1 to 4 38m Warning Unhealthy pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Startup probe failed: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Startup probe error: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused... 38m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-070e3d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 38m Normal ScalingReplicaSet deployment/e2e-inf-b-3e300c-openshift-default Scaled up replica set e2e-inf-b-3e300c-openshift-default-679f5f7777 from 4 to 8 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb 38m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-070e3d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3e300c-openshift-default New size: 8; reason: All metrics below target 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl 38m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Readiness probe failed: Get "http://10.134.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Readiness probe error: Get "http://10.134.0.140:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Readiness probe failed: Get "http://10.134.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 38m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Readiness probe error: Get "http://10.134.0.134:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 38m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Liveness probe failed: dial tcp 10.134.0.105:50052: i/o timeout 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-3e300c-openshift-default New size: 10; reason: All metrics below target 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 (combined from similar events): Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 38m Normal SuccessfulCreate replicaset/e2e-inf-b-3e300c-openshift-default-679f5f7777 Created pod: e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww 38m Normal ScalingReplicaSet deployment/e2e-inf-b-3e300c-openshift-default Scaled up replica set e2e-inf-b-3e300c-openshift-default-679f5f7777 from 8 to 10 38m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Readiness probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Readiness probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused 38m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Container created 38m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Container started 38m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Startup probe failed: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused 37m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Startup probe error: Get "http://10.134.0.134:15021/healthz/ready": dial tcp 10.134.0.134:15021: connect: connection refused... 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-070e3d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-070e3d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 37m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-3e300c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-3e300c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-070e3d-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 37m Normal SuccessfulCreate replicaset/e2e-inf-b-070e3d-openshift-default-68c88bd895 Created pod: e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k 37m Normal ScalingReplicaSet deployment/e2e-inf-b-070e3d-openshift-default Scaled up replica set e2e-inf-b-070e3d-openshift-default-68c88bd895 from 1 to 2 37m Normal Killing pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Stopping container payload-pre-processing 37m Normal Killing pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Stopping container payload-processing 37m Normal SuccessfulDelete replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Deleted pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 37m Normal SuccessfulDelete replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Deleted pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq 37m Normal SuccessfulDelete replicaset/e2e-inf-a-edd32a-openshift-default-7cf5f49d65 Deleted pod: e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp 37m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-hmkt5 Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-nkqsq Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mkhdp Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-edd32a-openshift-default New size: 4; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/e2e-inf-a-edd32a-openshift-default Scaled down replica set e2e-inf-a-edd32a-openshift-default-7cf5f49d65 from 7 to 4 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 37m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 37m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn Stopping container istio-proxy 37m Normal Created pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Container created 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm 37m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz Stopping container istio-proxy 37m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-kjqj5 Stopping container istio-proxy 37m Normal Pulled pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 37m Normal AddedInterface pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Add eth0 [10.133.0.151/23] from ovn-kubernetes 37m Normal ScalingReplicaSet deployment/e2e-worker-w2-f033b1-openshift-default Scaled down replica set e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 from 3 to 2 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default New size: 2; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-n4pvn 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default New size: 5; reason: All metrics below target 37m Normal Started pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Container started 37m Normal SuccessfulDelete replicaset/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9 Deleted pod: e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default New size: 5; reason: All metrics below target 37m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-sxkn5 Stopping container istio-proxy 37m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default Scaled down replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 6 to 5 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n 37m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-wwrcz 37m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 5; reason: All metrics below target 36m Warning Unhealthy pod/payload-processing-e2e-inf-a-edd32a-76686b7d6-tggwz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-edd32a-64b8bf5f55-hk28p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Add eth0 [10.132.0.239/23] from ovn-kubernetes 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Container created 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Add eth0 [10.132.0.240/23] from ovn-kubernetes 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Add eth0 [10.132.0.238/23] from ovn-kubernetes 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Container started 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Container started 36m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Readiness probe failed: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Container created 36m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-s4665 Readiness probe error: Get "http://10.134.0.56:15021/healthz/ready": dial tcp 10.134.0.56:15021: connect: connection refused... 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Container created 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Container started 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Container started 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Container created 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Add eth0 [10.134.0.143/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-mrhht Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-8vrhb Stopping container istio-proxy 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Container started 36m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-kgh82 Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Readiness probe failed: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Container created 36m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-8z97n Readiness probe error: Get "http://10.133.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Add eth0 [10.134.0.144/23] from ovn-kubernetes 36m Normal Killing pod/e2e-inf-a-edd32a-openshift-default-7cf5f49d65-szbgf Stopping container istio-proxy 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning Unhealthy pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Startup probe failed: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Startup probe error: Get "http://10.133.0.151:15021/healthz/ready": dial tcp 10.133.0.151:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 36m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-mr5tm Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Add eth0 [10.133.0.153/23] from ovn-kubernetes 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Add eth0 [10.133.0.152/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Container created 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Container started 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Add eth0 [10.133.0.154/23] from ovn-kubernetes 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Container created 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Container started 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Container created 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Container started 36m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-e7e2df-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Startup probe failed: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Startup probe failed: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Startup probe error: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Startup probe failed: Get "http://10.134.0.144:15021/healthz/ready": dial tcp 10.134.0.144:15021: connect: connection refused 36m Normal AddedInterface pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Add eth0 [10.134.0.145/23] from ovn-kubernetes 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Startup probe error: Get "http://10.134.0.143:15021/healthz/ready": dial tcp 10.134.0.143:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Startup probe error: Get "http://10.132.0.238:15021/healthz/ready": dial tcp 10.132.0.238:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Add eth0 [10.133.0.155/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Startup probe error: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Startup probe failed: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Startup probe error: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused... 36m Normal Pulled pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Add eth0 [10.132.0.242/23] from ovn-kubernetes 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Startup probe failed: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Startup probe error: Get "http://10.133.0.152:15021/healthz/ready": dial tcp 10.133.0.152:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Startup probe failed: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused 36m Normal Started pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Container started 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Startup probe failed: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused 36m Normal Created pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Container created 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Startup probe failed: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Startup probe error: Get "http://10.133.0.153:15021/healthz/ready": dial tcp 10.133.0.153:15021: connect: connection refused... 36m Normal Started pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Container started 36m Normal Created pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Container created 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Startup probe error: Get "http://10.133.0.154:15021/healthz/ready": dial tcp 10.133.0.154:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Add eth0 [10.132.0.243/23] from ovn-kubernetes 36m Normal Pulled pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Created pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Container created 36m Normal Started pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Container started 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-e7e2df Scaled up replica set payload-processing-e2e-inf-a-e7e2df-558bccd986 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-e7e2df-558bccd986 Created pod: payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-e7e2df Scaled up replica set payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-e7e2df-558bccd986 Error creating: pods "payload-processing-e2e-inf-a-e7e2df-558bccd986-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e7e2df: serviceaccount "payload-processing-e2e-inf-a-e7e2df" not found 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c Created pod: payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c Error creating: pods "payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-e7e2df: serviceaccount "payload-processing-e2e-inf-a-e7e2df" not found 36m Normal Created pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Container created 36m Normal Pulled pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Container started 36m Normal Created pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Container created 36m Normal AddedInterface pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Add eth0 [10.133.0.157/23] from ovn-kubernetes 36m Normal Pulled pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Started pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Container started 36m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Add eth0 [10.133.0.156/23] from ovn-kubernetes 36m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Startup probe failed: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Startup probe error: Get "http://10.132.0.242:15021/healthz/ready": dial tcp 10.132.0.242:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Startup probe failed: Get "http://10.132.0.242:15021/healthz/ready": dial tcp 10.132.0.242:15021: connect: connection refused 36m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Startup probe error: Get "http://10.133.0.155:15021/healthz/ready": dial tcp 10.133.0.155:15021: connect: connection refused... 36m Warning ProbeError pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Startup probe error: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused... 36m Warning Unhealthy pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Startup probe failed: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused 36m Normal Killing pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Stopping container payload-pre-processing 36m Normal Killing pod/e2e-inf-b-3577a0-openshift-default-84498fd47-8gngc Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-3577a0-openshift-default-84498fd47-bqmq4 Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Stopping container istio-proxy 36m Normal AddedLabel gateway/e2e-shared-a-w6-d6e25f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-d6e25f 36m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-d6e25f-openshift-default Scaled up replica set e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 Created pod: e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 36m Warning ProbeError pod/e2e-inf-b-3577a0-openshift-default-84498fd47-88jkv Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal AddedInterface pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Add eth0 [10.134.0.146/23] from ovn-kubernetes 36m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c Error creating: pods "payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-d6e25f: serviceaccount "payload-processing-e2e-shared-a-w6-d6e25f" not found 36m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-d6e25f Scaled up replica set payload-processing-e2e-shared-a-w6-d6e25f-dc644b8 from 0 to 1 36m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c Created pod: payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 36m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-d6e25f Scaled up replica set payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c from 0 to 1 36m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8 Error creating: pods "payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-d6e25f: serviceaccount "payload-processing-e2e-shared-a-w6-d6e25f" not found 36m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8 Created pod: payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn 36m Normal Created pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Container created 36m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Container created 36m Normal Started pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Container started 36m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Container started 36m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal Pulled pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Add eth0 [10.133.0.160/23] from ovn-kubernetes 36m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Add eth0 [10.133.0.159/23] from ovn-kubernetes 36m Warning Unhealthy pod/payload-processing-e2e-inf-b-3577a0-d87647984-9xrbz Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-3577a0-f5df89dbc-fvvrx Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-9d8b5d-openshift-default New size: 3; reason: All metrics below target 36m Normal Killing pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-inf-b-830149-openshift-default-64cc8cfc59 Deleted pod: e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-830149-openshift-default New size: 8; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-inf-b-830149-openshift-default Scaled down replica set e2e-inf-b-830149-openshift-default-64cc8cfc59 from 9 to 8 36m Normal ScalingReplicaSet deployment/e2e-inf-b-9d8b5d-openshift-default Scaled down replica set e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc from 5 to 3 36m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Deleted pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm 36m Normal SuccessfulDelete replicaset/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc Deleted pod: e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-tzkng 36m Normal Killing pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Deleted pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq 36m Normal SuccessfulDelete replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Deleted pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default New size: 7; reason: All metrics below target 36m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled down replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 9 to 7 36m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-z4dnq Stopping container istio-proxy 36m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwwkz Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Readiness probe failed: HTTP probe failed with statuscode: 503 36m Warning ProbeError pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-n9fwm Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default New size: 4; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl 36m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-dj75l 36m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-lrtdl Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 36m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-ddfc4 Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default New size: 3; reason: All metrics below target 36m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6grvg Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal SuccessfulCreate replicaset/e2e-inf-b-6b4c49-openshift-default-656f9585df Created pod: e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 35m Normal ScalingReplicaSet deployment/e2e-inf-b-6b4c49-openshift-default Scaled up replica set e2e-inf-b-6b4c49-openshift-default-656f9585df from 0 to 1 35m Normal AddedLabel gateway/e2e-inf-b-6b4c49 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-6b4c49 35m Normal AddedInterface pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Add eth0 [10.132.0.244/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-6b4c49 Scaled up replica set payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f from 0 to 1 35m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-6b4c49-64cdb676c4 Error creating: pods "payload-processing-e2e-inf-b-6b4c49-64cdb676c4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6b4c49: serviceaccount "payload-processing-e2e-inf-b-6b4c49" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f Error creating: pods "payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-6b4c49: serviceaccount "payload-processing-e2e-inf-b-6b4c49" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-6b4c49 Scaled up replica set payload-processing-e2e-inf-b-6b4c49-64cdb676c4 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-6b4c49-64cdb676c4 Created pod: payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f Created pod: payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l 35m Normal Started pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Container started 35m Normal Created pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Container created 35m Normal Pulled pod/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf Add eth0 [10.134.0.148/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf Container started 35m Normal Pulled pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Add eth0 [10.134.0.147/23] from ovn-kubernetes 35m Normal Created pod/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf Container created 35m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled down replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 7 to 5 35m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Deleted pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz 35m Normal SuccessfulDelete replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Deleted pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 35m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-55jsz Stopping container istio-proxy 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default New size: 5; reason: All metrics below target 35m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-47sm4 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal ScalingReplicaSet deployment/e2e-inf-b-8483c2-openshift-default Scaled up replica set e2e-inf-b-8483c2-openshift-default-86687957bf from 1 to 2 35m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-8483c2-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 35m Normal SuccessfulCreate replicaset/e2e-inf-b-8483c2-openshift-default-86687957bf Created pod: e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s 35m Normal Pulled pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Add eth0 [10.133.0.167/23] from ovn-kubernetes 35m Normal Created pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Container created 35m Normal Started pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Container started 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-c8vgk 35m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-c8vgk Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 10 to 7 35m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-vwrt5 Stopping container istio-proxy 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-vwrt5 35m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-vktwt 35m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-vktwt Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Startup probe failed: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Startup probe error: Get "http://10.133.0.167:15021/healthz/ready": dial tcp 10.133.0.167:15021: connect: connection refused... 35m Normal AddedLabel gateway/e2e-shared-b-w6-da34ca Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-da34ca 35m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-da34ca-openshift-default Scaled up replica set e2e-shared-b-w6-da34ca-openshift-default-6db76b45c from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c Created pod: e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c 35m Normal AddedInterface pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Add eth0 [10.133.0.170/23] from ovn-kubernetes 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4 Error creating: pods "payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-da34ca: serviceaccount "payload-processing-e2e-shared-b-w6-da34ca" not found 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-da34ca Scaled up replica set payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4 Created pod: payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-da34ca Scaled up replica set payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67 Created pod: payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67 Error creating: pods "payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-da34ca: serviceaccount "payload-processing-e2e-shared-b-w6-da34ca" not found 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Add eth0 [10.133.0.172/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Add eth0 [10.133.0.171/23] from ovn-kubernetes 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Container started 35m Normal Pulled pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Container created 35m Normal Started pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Container started 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-a-d0a5de-5989495d4b-mfgmv Stopping container payload-processing 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-f2760d-577bdfbb74-6lb25 Readiness probe failed: timeout: failed to connect service "10.134.0.119:50052" within 1s: context deadline exceeded 34m Normal Pulled pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Container started 34m Normal Created pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Container created 34m Normal Killing pod/e2e-inf-a-d0a5de-openshift-default-79f4bc7977-l9bjv Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-s7dbt Stopping container istio-proxy 34m Warning ProbeError pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Startup probe error: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused... 34m Normal Killing pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Stopping container payload-processing 34m Warning Unhealthy pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Startup probe failed: Get "http://10.134.0.146:15021/healthz/ready": dial tcp 10.134.0.146:15021: connect: connection refused 34m Normal Killing pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Stopping container payload-pre-processing 34m Warning ProbeError pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Readiness probe error: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused... 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-f2760d-d96bc568c-9grkw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/e2e-inf-a-f2760d-openshift-default-845f578cdf-qwc8c Readiness probe failed: Get "http://10.132.0.243:15021/healthz/ready": dial tcp 10.132.0.243:15021: connect: connection refused 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-b-9d8b5d-76d68746b6-7s7mh Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Stopping container payload-pre-processing 34m Normal Killing pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Stopping container payload-processing 34m Normal ScalingReplicaSet deployment/e2e-inf-a-dfdb11-openshift-default Scaled up replica set e2e-inf-a-dfdb11-openshift-default-75fbd6bd67 from 1 to 2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-dfdb11-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal SuccessfulCreate replicaset/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67 Created pod: e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 34m Normal AddedInterface pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Add eth0 [10.133.0.174/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Container started 34m Normal Created pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Container created 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-d0a5de-7fbff57d8c-zz2kl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Startup probe failed: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Startup probe error: Get "http://10.133.0.174:15021/healthz/ready": dial tcp 10.133.0.174:15021: connect: connection refused... 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-17957f-openshift-default New size: 4; reason: All metrics below target 34m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Stopping container istio-proxy 34m Normal ScalingReplicaSet deployment/e2e-inf-a-17957f-openshift-default Scaled down replica set e2e-inf-a-17957f-openshift-default-64b4cbb957 from 5 to 4 34m Normal SuccessfulDelete replicaset/e2e-inf-a-17957f-openshift-default-64b4cbb957 Deleted pod: e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-17957f-86785bdb55-vvrzj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-gwpjt Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning Unhealthy pod/payload-processing-e2e-inf-a-17957f-5686759fdb-fcrcm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-processing-e2e-inf-b-8483c2-57fd7d7846-mgl92 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-8483c2-79469b67b7-s88jw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-7bg9d Stopping container istio-proxy 34m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 34m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 7 to 6 34m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-7bg9d 34m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-fb2cw Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-544w5 Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-7wf2m Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-9d8b5d-openshift-default-6f9cdb5bcc-qrmdd Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-5qbdz Stopping container istio-proxy 34m Normal Killing pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-tg8sr Stopping container istio-proxy 34m Normal AddedLabel gateway/e2e-shared-a-w1-e3890e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-e3890e 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-e3890e-openshift-default Scaled up replica set e2e-shared-a-w1-e3890e-openshift-default-6975b574b5 from 0 to 1 34m Normal Killing pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Stopping container istio-proxy 34m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-9d8b5d-64f95b8d99-n9vp9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 34m Warning Unhealthy pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Readiness probe failed: HTTP probe failed with statuscode: 503 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5 Created pod: e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l 34m Normal AddedInterface pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Add eth0 [10.134.0.155/23] from ovn-kubernetes 34m Warning Unhealthy pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Readiness probe failed: HTTP probe failed with statuscode: 503 34m Warning ProbeError pod/e2e-inf-b-8483c2-openshift-default-86687957bf-zhgcm Readiness probe error: HTTP probe failed with statuscode: 503... 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-d6e25f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-d6e25f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Warning ProbeError pod/e2e-inf-b-8483c2-openshift-default-86687957bf-hxs6s Readiness probe error: HTTP probe failed with statuscode: 503... 34m Normal Created pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Container created 34m Warning ProbeError pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Readiness probe error: Get "http://10.134.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-e3890e Scaled up replica set payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67 from 0 to 1 34m Normal Started pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Container started 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-e3890e Scaled up replica set payload-processing-e2e-shared-a-w1-e3890e-8689f59474 from 0 to 1 34m Warning Unhealthy pod/e2e-inf-a-17957f-openshift-default-64b4cbb957-mz8qp Readiness probe failed: Get "http://10.134.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Normal Pulled pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Container created 34m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Add eth0 [10.134.0.157/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67 Created pod: payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-e3890e-8689f59474 Error creating: pods "payload-processing-e2e-shared-a-w1-e3890e-8689f59474-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-e3890e: serviceaccount "payload-processing-e2e-shared-a-w1-e3890e" not found 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-e3890e: serviceaccount "payload-processing-e2e-shared-a-w1-e3890e" not found 34m Normal Started pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Container started 34m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Container started 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-e3890e-8689f59474 Created pod: payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm 34m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Container created 34m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Add eth0 [10.134.0.156/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Killing pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Stopping container payload-processing 34m Normal Killing pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Stopping container payload-pre-processing 34m Warning ProbeError pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Startup probe error: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Startup probe failed: Get "http://10.134.0.145:15021/healthz/ready": dial tcp 10.134.0.145:15021: connect: connection refused 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-e7e2df-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-vxr5s Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-9lksb Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-2pqqz Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-972dx Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-774ww Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Stopping container istio-proxy 33m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-dvm2p Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/payload-processing-e2e-inf-b-830149-7f864c9c64-rvq4m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-830149-58d9646cff-lnzrg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-t8bqt Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-inf-b-830149-openshift-default-64cc8cfc59-6tf6f Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Normal Killing pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Stopping container payload-processing 33m Normal Killing pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Stopping container payload-pre-processing 33m Normal AddedLabel gateway/e2e-shared-b-w1-51e3b7 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-51e3b7 33m Normal AddedInterface pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Add eth0 [10.134.0.159/23] from ovn-kubernetes 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-51e3b7-openshift-default Scaled up replica set e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc from 0 to 1 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc Created pod: e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj 33m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf Error creating: pods "payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-51e3b7: serviceaccount "payload-processing-e2e-shared-b-w1-51e3b7" not found 33m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf Created pod: payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h 33m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-51e3b7 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf from 0 to 1 33m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z 33m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 Stopping container istio-proxy 33m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-qpqg9 33m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d Error creating: pods "payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-51e3b7: serviceaccount "payload-processing-e2e-shared-b-w1-51e3b7" not found 33m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d Created pod: payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm 33m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-51e3b7 Scaled up replica set payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d from 0 to 1 33m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v 33m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Stopping container istio-proxy 33m Normal Started pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Container started 33m Normal Created pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Container created 33m Normal Pulled pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Add eth0 [10.133.0.175/23] from ovn-kubernetes 33m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Container started 33m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Stopping container istio-proxy 33m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Container created 33m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Add eth0 [10.134.0.160/23] from ovn-kubernetes 33m Normal Killing pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-hcfbd Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-64bw8 Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Stopping container istio-proxy 33m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-9px6z Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/payload-processing-e2e-inf-a-487639-cdbf7c87d-7txk8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Readiness probe failed: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused 33m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-487639-6bc59bb99f-nxltv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 33m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5j56v Readiness probe error: Get "http://10.133.0.60:15021/healthz/ready": dial tcp 10.133.0.60:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-ctbrg Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning ProbeError pod/e2e-inf-a-487639-openshift-default-5d5fc9449c-7jtfw Readiness probe error: Get "http://10.132.0.221:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal Created pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Container created 33m Normal Killing pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Stopping container payload-pre-processing 33m Normal Pulled pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Killing pod/payload-processing-e2e-inf-b-3e300c-74b58496db-grjgh Stopping container payload-processing 33m Normal Started pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Container started 33m Normal Pulled pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Container started 33m Normal Created pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Container created 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-da34ca-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-da34ca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-6b4c49-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-6b4c49-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Startup probe failed: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Startup probe error: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Startup probe error: Get "http://10.132.0.244:15021/healthz/ready": dial tcp 10.132.0.244:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Startup probe failed: Get "http://10.133.0.170:15021/healthz/ready": dial tcp 10.133.0.170:15021: connect: connection refused 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9 Created pod: e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w2-702002-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-702002-openshift-default Scaled up replica set e2e-shared-a-w2-702002-openshift-default-5588c5c5d9 from 1 to 2 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-1783e3-openshift-default Scaled up replica set e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc from 1 to 4 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal AddedInterface pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Add eth0 [10.134.0.161/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-e3890e-openshift-default Scaled up replica set e2e-shared-a-w1-e3890e-openshift-default-6975b574b5 from 1 to 4 33m Normal AddedInterface pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Add eth0 [10.134.0.163/23] from ovn-kubernetes 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5 Created pod: e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g 33m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Container started 33m Normal Pulled pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Pulled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Container created 33m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-d6e25f-openshift-default Scaled up replica set e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 from 1 to 2 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-d6e25f-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5 Created pod: e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5 Created pod: e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 33m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Add eth0 [10.133.0.177/23] from ovn-kubernetes 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 Created pod: e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 33m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Container started 33m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Add eth0 [10.134.0.162/23] from ovn-kubernetes 33m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Container created 33m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Add eth0 [10.132.1.27/23] from ovn-kubernetes 33m Normal AddedInterface pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Add eth0 [10.132.1.28/23] from ovn-kubernetes 33m Normal Pulled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Add eth0 [10.133.0.178/23] from ovn-kubernetes 33m Normal Created pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Container created 33m Normal Created pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Container created 33m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Container started 33m Normal Started pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Container started 33m Normal Created pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Container created 33m Normal Created pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Container created 33m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Container created 33m Normal Started pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Container started 33m Normal Started pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Container started 33m Normal Started pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Container started 33m Normal Started pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Container started 33m Normal Created pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Container created 33m Normal Pulled pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal AddedInterface pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Add eth0 [10.132.1.29/23] from ovn-kubernetes 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-dxhsm Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Stopping container istio-proxy 33m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-521734-openshift-default Scaled up replica set e2e-shared-a-w3-521734-openshift-default-756d487fd9 from 0 to 1 33m Normal AddedLabel gateway/e2e-shared-a-w3-521734 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-521734 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Stopping container istio-proxy 33m Normal Killing pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-pt6z6 Stopping container istio-proxy 33m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq 33m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-c5sn2 Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8db5c Readiness probe error: HTTP probe failed with statuscode: 503... 33m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Readiness probe failed: HTTP probe failed with statuscode: 503 33m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-4r72d Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-ckft2 Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-fzklb Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Readiness probe failed: HTTP probe failed with statuscode: 503 32m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-wmpww Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning ProbeError pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-bmhzl Readiness probe error: HTTP probe failed with statuscode: 503... 32m Warning Unhealthy pod/e2e-inf-b-3e300c-openshift-default-679f5f7777-8r6rf Readiness probe failed: HTTP probe failed with statuscode: 503 32m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Add eth0 [10.134.0.164/23] from ovn-kubernetes 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Startup probe error: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Startup probe failed: Get "http://10.133.0.178:15021/healthz/ready": dial tcp 10.133.0.178:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Startup probe failed: Get "http://10.132.1.28:15021/healthz/ready": dial tcp 10.132.1.28:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Startup probe error: Get "http://10.132.1.28:15021/healthz/ready": dial tcp 10.132.1.28:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Container created 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Startup probe failed: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused 32m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-3e300c-6ddd474b99-hq8np Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Container started 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Startup probe error: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Startup probe failed: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Startup probe error: Get "http://10.134.0.161:15021/healthz/ready": dial tcp 10.134.0.161:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Startup probe error: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Startup probe failed: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-521734 Scaled up replica set payload-pre-processing-e2e-shared-a-w3-521734-f8998689f from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f Error creating: pods "payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-521734: serviceaccount "payload-processing-e2e-shared-a-w3-521734" not found 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-521734 Scaled up replica set payload-processing-e2e-shared-a-w3-521734-867d896c58 from 0 to 1 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-521734-867d896c58 Created pod: payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-521734-867d896c58 Error creating: pods "payload-processing-e2e-shared-a-w3-521734-867d896c58-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-521734: serviceaccount "payload-processing-e2e-shared-a-w3-521734" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f Created pod: payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Pulled pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Add eth0 [10.133.0.180/23] from ovn-kubernetes 32m Normal Created pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Container created 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Add eth0 [10.133.0.179/23] from ovn-kubernetes 32m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Container created 32m Normal Started pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Container started 32m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Container started 32m Normal Pulled pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Startup probe error: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused... 32m Normal Created pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Container created 32m Normal Started pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Container started 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Startup probe failed: Get "http://10.134.0.79:15021/healthz/ready": dial tcp 10.134.0.79:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Startup probe error: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Startup probe failed: Get "http://10.134.0.155:15021/healthz/ready": dial tcp 10.134.0.155:15021: connect: connection refused 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Container started 32m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Add eth0 [10.132.1.32/23] from ovn-kubernetes 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w 32m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Add eth0 [10.133.0.181/23] from ovn-kubernetes 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Add eth0 [10.134.0.165/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Container created 32m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-1783e3-openshift-default Scaled up replica set e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc from 4 to 8 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Container started 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Container created 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl 32m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Add eth0 [10.133.0.182/23] from ovn-kubernetes 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Container started 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Container created 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Container started 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Container created 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-da34ca-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-da34ca-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-6b4c49-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Startup probe error: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused... 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Startup probe error: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Startup probe failed: Get "http://10.132.1.32:15021/healthz/ready": dial tcp 10.132.1.32:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Startup probe failed: Get "http://10.134.0.165:15021/healthz/ready": dial tcp 10.134.0.165:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Startup probe failed: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Startup probe error: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused... 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-6b4c49-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default New size: 10; reason: All metrics below target 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 32m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Container created 32m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Container created 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Container started 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Add eth0 [10.133.0.183/23] from ovn-kubernetes 32m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Add eth0 [10.132.1.34/23] from ovn-kubernetes 32m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Container started 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Startup probe error: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused... 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Startup probe failed: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused 32m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Startup probe failed: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Startup probe error: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused... 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-d2832c-openshift-default Scaled up replica set e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567 from 0 to 1 32m Normal AddedLabel gateway/e2e-shared-b-w3-d2832c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-d2832c 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567 Created pod: e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb 32m Normal AddedInterface pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Add eth0 [10.134.0.166/23] from ovn-kubernetes 32m Normal Created pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Container created 32m Normal Created pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Container created 32m Normal Started pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Container started 32m Normal Pulled pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal Started pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Container started 32m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd Error creating: pods "payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d2832c: serviceaccount "payload-processing-e2e-shared-b-w3-d2832c" not found 32m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd Created pod: payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x 32m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-d2832c Scaled up replica set payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd from 0 to 1 32m Normal Pulled pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Add eth0 [10.134.0.168/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-d2832c Scaled up replica set payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5 from 0 to 1 32m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5 Error creating: pods "payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-d2832c: serviceaccount "payload-processing-e2e-shared-b-w3-d2832c" not found 32m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5 Created pod: payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-e2bf63-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-e2bf63-openshift-default Scaled up replica set e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 from 1 to 2 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 Created pod: e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz 32m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Container started 32m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Container created 32m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Add eth0 [10.134.0.167/23] from ovn-kubernetes 32m Normal Started pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Container started 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Created pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Container created 32m Normal Pulled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal AddedInterface pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Add eth0 [10.133.0.184/23] from ovn-kubernetes 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal AddedInterface pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Add eth0 [10.132.1.35/23] from ovn-kubernetes 32m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-de0cba-openshift-default Scaled up replica set e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7 from 1 to 2 32m Normal Pulled pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w2-de0cba-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 32m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7 Created pod: e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq 32m Normal Created pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Container created 32m Normal Started pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Container started 31m Warning Unhealthy pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Startup probe failed: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Startup probe error: Get "http://10.134.0.159:15021/healthz/ready": dial tcp 10.134.0.159:15021: connect: connection refused... 31m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Startup probe error: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Startup probe failed: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Startup probe error: Get "http://10.132.1.35:15021/healthz/ready": dial tcp 10.132.1.35:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Startup probe failed: Get "http://10.132.1.35:15021/healthz/ready": dial tcp 10.132.1.35:15021: connect: connection refused 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc Created pod: e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-51e3b7-openshift-default Scaled up replica set e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc from 1 to 2 31m Normal Pulled pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Add eth0 [10.132.1.40/23] from ovn-kubernetes 31m Normal Started pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Container started 31m Normal Created pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Container created 31m Normal Killing pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Stopping container payload-processing 31m Normal Killing pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Stopping container payload-pre-processing 31m Warning ProbeError pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Startup probe error: Get "http://10.132.1.40:15021/healthz/ready": dial tcp 10.132.1.40:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Startup probe failed: Get "http://10.132.1.40:15021/healthz/ready": dial tcp 10.132.1.40:15021: connect: connection refused 31m Normal Killing pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Stopping container istio-proxy 31m Warning ProbeError pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-dggzs Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning ProbeError pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Readiness probe error: HTTP probe failed with statuscode: 503... 31m Warning Unhealthy pod/e2e-inf-a-dfdb11-openshift-default-75fbd6bd67-ldt55 Readiness probe failed: HTTP probe failed with statuscode: 503 31m Warning Unhealthy pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-b-070e3d-6b97d99dd-8zshf Stopping container payload-processing 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-dfdb11-7c78bdc4dc-fkzr7 Readiness probe failed: timeout: failed to connect service "10.134.0.132:50052" within 1s: context deadline exceeded 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-dfdb11-fd9d47844-gc6g4 Readiness probe failed: timeout: failed to connect service "10.134.0.133:50052" within 1s: context deadline exceeded 30m Normal Killing pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-hqx8k Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Stopping container istio-proxy 30m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 30m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 6 to 5 30m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-9sjh9 Stopping container istio-proxy 30m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-9sjh9 30m Warning Unhealthy pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Readiness probe failed: HTTP probe failed with statuscode: 503 30m Warning ProbeError pod/e2e-inf-b-070e3d-openshift-default-68c88bd895-zsdg2 Readiness probe error: HTTP probe failed with statuscode: 503... 30m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 Created pod: e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j 30m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-d6e25f-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 30m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-d6e25f-openshift-default Scaled up replica set e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 from 2 to 3 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-070e3d-5448b9d84c-t6vp9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Pulled pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 30m Normal AddedInterface pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Add eth0 [10.133.0.185/23] from ovn-kubernetes 30m Normal Created pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Container created 30m Normal Started pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Container started 30m Warning ProbeError pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Startup probe error: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused... 30m Warning Unhealthy pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Startup probe failed: Get "http://10.133.0.185:15021/healthz/ready": dial tcp 10.133.0.185:15021: connect: connection refused 29m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Created pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Container created 29m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k 29m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Add eth0 [10.133.0.186/23] from ovn-kubernetes 29m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Add eth0 [10.133.0.187/23] from ovn-kubernetes 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-521734-openshift-default Scaled up replica set e2e-shared-a-w3-521734-openshift-default-756d487fd9 from 1 to 4 29m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Add eth0 [10.132.1.46/23] from ovn-kubernetes 29m Normal Pulled pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal Started pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Container started 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l 29m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Container started 29m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Container created 29m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Container started 29m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Container created 29m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Container created 29m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Container started 29m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Container started 29m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Container created 29m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Startup probe failed: Get "http://10.132.1.46:15021/healthz/ready": dial tcp 10.132.1.46:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Startup probe error: Get "http://10.132.1.46:15021/healthz/ready": dial tcp 10.132.1.46:15021: connect: connection refused... 29m Warning ProbeError pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Startup probe error: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Startup probe failed: Get "http://10.134.0.166:15021/healthz/ready": dial tcp 10.134.0.166:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Startup probe error: Get "http://10.133.0.186:15021/healthz/ready": dial tcp 10.133.0.186:15021: connect: connection refused... 29m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Startup probe error: Get "http://10.133.0.187:15021/healthz/ready": dial tcp 10.133.0.187:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Startup probe failed: Get "http://10.133.0.187:15021/healthz/ready": dial tcp 10.133.0.187:15021: connect: connection refused 29m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Startup probe failed: Get "http://10.133.0.186:15021/healthz/ready": dial tcp 10.133.0.186:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Startup probe error: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused... 29m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Startup probe failed: Get "http://10.134.0.164:15021/healthz/ready": dial tcp 10.134.0.164:15021: connect: connection refused 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 Deleted pod: e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-b0b7a3-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-b0b7a3-openshift-default-555bd95f85 from 3 to 2 29m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-zzt5p Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-521734-openshift-default Scaled up replica set e2e-shared-a-w3-521734-openshift-default-756d487fd9 from 4 to 8 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 29m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Add eth0 [10.134.0.171/23] from ovn-kubernetes 29m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-782nd 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default New size: 8; reason: All metrics below target 29m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 29m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Stopping container discovery 28m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Container created 28m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Add eth0 [10.133.0.188/23] from ovn-kubernetes 28m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Container started 28m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal SuccessfulDelete replicaset/e2e-worker-w1-53b784-openshift-default-7d967d7c57 Deleted pod: e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv 28m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Container started 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 3 to 2 28m Normal SuccessfulDelete replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Deleted pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w 28m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Add eth0 [10.132.1.47/23] from ovn-kubernetes 28m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Stopping container istio-proxy 28m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Container created 28m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Container started 28m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Container created 28m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default (combined from similar events): New size: 3; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w3-ff3e70-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-ff3e70-openshift-default-68794df669 from 4 to 3 28m Normal SuccessfulDelete replicaset/e2e-worker-w3-ff3e70-openshift-default-68794df669 Deleted pod: e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 28m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Container created 28m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Add eth0 [10.132.1.48/23] from ovn-kubernetes 28m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Container started 28m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Stopping container istio-proxy 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w1-53b784-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-53b784-openshift-default-7d967d7c57 from 3 to 2 28m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-l89vv Stopping container istio-proxy 28m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Readiness probe error: HTTP probe failed with statuscode: 503... 28m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Readiness probe error: Get "http://10.133.0.93:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 28m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4ff52 Readiness probe failed: HTTP probe failed with statuscode: 503 28m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-782nd Readiness probe failed: Get "http://10.133.0.93:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 28m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Readiness probe error: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Startup probe error: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-cjm6w Readiness probe failed: Get "http://10.132.0.75:15021/healthz/ready": dial tcp 10.132.0.75:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Startup probe failed: Get "http://10.133.0.188:15021/healthz/ready": dial tcp 10.133.0.188:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Startup probe failed: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Startup probe error: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused... 28m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Startup probe error: Get "http://10.132.1.48:15021/healthz/ready": dial tcp 10.132.1.48:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Startup probe failed: Get "http://10.132.1.47:15021/healthz/ready": dial tcp 10.132.1.47:15021: connect: connection refused 28m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Startup probe failed: Get "http://10.132.1.48:15021/healthz/ready": dial tcp 10.132.1.48:15021: connect: connection refused 28m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Startup probe error: Get "http://10.132.1.47:15021/healthz/ready": dial tcp 10.132.1.47:15021: connect: connection refused... 28m Normal ScalingReplicaSet deployment/e2e-worker-w4-0def36-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-0def36-openshift-default-7d684697cc from 3 to 2 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal SuccessfulDelete replicaset/e2e-worker-w4-0def36-openshift-default-7d684697cc Deleted pod: e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr 28m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-fdxrr Stopping container istio-proxy 28m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm Stopping container istio-proxy 28m Normal SuccessfulDelete replicaset/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 Deleted pod: e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-s2xvm 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 28m Normal ScalingReplicaSet deployment/e2e-worker-w0-1f3a6b-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-1f3a6b-openshift-default-db74bcc99 from 3 to 2 28m Normal AddedInterface pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Add eth0 [10.134.0.172/23] from ovn-kubernetes 28m Normal Created pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Container created 28m Normal Pulled pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal Started pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Container started 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-da34ca-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c Created pod: e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s 28m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-da34ca-openshift-default Scaled up replica set e2e-shared-b-w6-da34ca-openshift-default-6db76b45c from 1 to 2 28m Normal AddedInterface pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Add eth0 [10.133.0.190/23] from ovn-kubernetes 28m Normal Created pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Container created 28m Normal Started pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Container started 28m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-6b4c49-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 28m Normal SuccessfulCreate replicaset/e2e-inf-b-6b4c49-openshift-default-656f9585df Created pod: e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw 28m Normal Pulled pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Normal ScalingReplicaSet deployment/e2e-inf-b-6b4c49-openshift-default Scaled up replica set e2e-inf-b-6b4c49-openshift-default-656f9585df from 1 to 2 28m Warning ProbeError pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 28m Warning ProbeError pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Startup probe error: Get "http://10.133.0.190:15021/healthz/ready": dial tcp 10.133.0.190:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Startup probe failed: Get "http://10.133.0.190:15021/healthz/ready": dial tcp 10.133.0.190:15021: connect: connection refused 28m Normal AddedInterface pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Add eth0 [10.133.0.191/23] from ovn-kubernetes 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-5a355b-openshift-default-8576bb9c45 Created pod: e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-5a355b-openshift-default Scaled up replica set e2e-ipp-a-5a355b-openshift-default-8576bb9c45 from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-a-5a355b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-5a355b 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4 Created pod: payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-5a355b-546989879f Error creating: pods "payload-pre-processing-e2e-ipp-a-5a355b-546989879f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5a355b: serviceaccount "payload-processing-e2e-ipp-a-5a355b" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-5a355b-546989879f Created pod: payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-5a355b Scaled up replica set payload-pre-processing-e2e-ipp-a-5a355b-546989879f from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4 Error creating: pods "payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5a355b: serviceaccount "payload-processing-e2e-ipp-a-5a355b" not found 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-5a355b Scaled up replica set payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4 from 0 to 1 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg Container started 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg Add eth0 [10.133.0.193/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw Container started 28m Normal Created pod/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw Container created 28m Normal Pulled pod/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw Add eth0 [10.133.0.192/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-e25317-openshift-default Scaled up replica set e2e-ipp-a-e25317-openshift-default-58f4485c6f from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz 27m Normal AddedLabel gateway/e2e-ipp-a-e25317 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e25317 27m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Add eth0 [10.134.0.173/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e25317 Scaled up replica set payload-processing-e2e-ipp-a-e25317-6d84998bb from 0 to 1 27m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-5wxxh Stopping container istio-proxy 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575 Error creating: pods "payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e25317: serviceaccount "payload-processing-e2e-ipp-a-e25317" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575 Created pod: payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e25317 Scaled up replica set payload-pre-processing-e2e-ipp-a-e25317-dc6d48575 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e25317-6d84998bb Error creating: pods "payload-processing-e2e-ipp-a-e25317-6d84998bb-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e25317: serviceaccount "payload-processing-e2e-ipp-a-e25317" not found 27m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 5 to 4 27m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-5wxxh 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e25317-6d84998bb Created pod: payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk 27m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 27m Normal Started pod/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk Container started 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq Container started 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq Add eth0 [10.134.0.175/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Pulled pod/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk Add eth0 [10.134.0.174/23] from ovn-kubernetes 27m Normal Created pod/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk Container created 27m Normal SuccessfulDelete replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Deleted pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 27m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-szlf4 Stopping container istio-proxy 27m Normal Killing pod/payload-pre-processing-e2e-ipp-a-5a355b-546989879f-jjvgg Stopping container payload-pre-processing 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default New size: 9; reason: All metrics below target 27m Normal Killing pod/payload-processing-e2e-ipp-a-5a355b-67b6d8b7f4-j4skw Stopping container payload-processing 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-1783e3-openshift-default Scaled down replica set e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc from 10 to 9 27m Normal Killing pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Stopping container payload-processing 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987 Error creating: pods "payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5a355b: serviceaccount "payload-processing-e2e-ipp-a-5a355b" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987 Created pod: payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-5a355b-596cd847f6 Error creating: pods "payload-processing-e2e-ipp-a-5a355b-596cd847f6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-5a355b: serviceaccount "payload-processing-e2e-ipp-a-5a355b" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-5a355b-596cd847f6 Created pod: payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-5a355b Scaled up replica set payload-processing-e2e-ipp-a-5a355b-596cd847f6 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-5a355b Scaled up replica set payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987 from 0 to 1 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Add eth0 [10.133.0.197/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Add eth0 [10.133.0.196/23] from ovn-kubernetes 27m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 27m Normal Pulling pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 27m Normal Killing pod/e2e-inf-a-e7e2df-openshift-default-584f54d89c-pwp7z Stopping container istio-proxy 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Container created 27m Normal Started pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Container started 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.292s (2.292s including waiting). Image size: 166620835 bytes. 27m Normal Pulled pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.301s (2.301s including waiting). Image size: 166620835 bytes. 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Container created 27m Normal Killing pod/payload-processing-e2e-ipp-a-e25317-6d84998bb-2gwsk Stopping container payload-processing 27m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e25317-dc6d48575-n4lpq Stopping container payload-pre-processing 27m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-e7e2df-69cf56d76c-5lf72 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Warning Unhealthy pod/payload-processing-e2e-inf-a-e7e2df-558bccd986-zktt6 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 27m Normal Pulled pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Container created 27m Normal Killing pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Stopping container payload-pre-processing 27m Normal Started pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Container started 27m Normal Killing pod/payload-processing-e2e-inf-b-6b4c49-64cdb676c4-pq9nf Stopping container payload-processing 26m Warning ProbeError pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Startup probe error: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Startup probe failed: Get "http://10.133.0.191:15021/healthz/ready": dial tcp 10.133.0.191:15021: connect: connection refused 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-ad047b-openshift-default Scaled up replica set e2e-ipp-b-ad047b-openshift-default-76dc48b85 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e25317 Scaled up replica set payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e25317 Scaled up replica set payload-processing-e2e-ipp-a-e25317-7bcfcd9d96 from 0 to 1 26m Normal AddedLabel gateway/e2e-ipp-b-ad047b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-ad047b 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96 Error creating: pods "payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e25317: serviceaccount "payload-processing-e2e-ipp-a-e25317" not found 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-ad047b-openshift-default-76dc48b85 Created pod: e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m Add eth0 [10.133.0.198/23] from ovn-kubernetes 26m Normal Pulling pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m Container created 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m Container started 26m Normal AddedInterface pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Add eth0 [10.134.0.176/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75 Error creating: pods "payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e25317: serviceaccount "payload-processing-e2e-ipp-a-e25317" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75 Created pod: payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Add eth0 [10.134.0.177/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96 Created pod: payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb 26m Normal Created pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.275s (2.275s including waiting). Image size: 166620835 bytes. 26m Normal Started pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Container started 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-ad047b Scaled up replica set payload-processing-e2e-ipp-b-ad047b-678ff75974 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-ad047b Scaled up replica set payload-pre-processing-e2e-ipp-b-ad047b-54f599749f from 0 to 1 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm Add eth0 [10.133.0.199/23] from ovn-kubernetes 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq Add eth0 [10.133.0.200/23] from ovn-kubernetes 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-ad047b-678ff75974 Created pod: payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq 26m Normal Pulled pod/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-ad047b-678ff75974 Error creating: pods "payload-processing-e2e-ipp-b-ad047b-678ff75974-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ad047b: serviceaccount "payload-processing-e2e-ipp-b-ad047b" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f Created pod: payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f Error creating: pods "payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ad047b: serviceaccount "payload-processing-e2e-ipp-b-ad047b" not found 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm Container created 26m Normal Created pod/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq Container created 26m Normal Started pod/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq Container started 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-5a355b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-5a355b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-d2hv2 Stopping container istio-proxy 26m Normal Killing pod/e2e-inf-b-6b4c49-openshift-default-656f9585df-w6zfw Stopping container istio-proxy 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-09656e-openshift-default-6d5c46dfc Created pod: e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-09656e-openshift-default Scaled up replica set e2e-ipp-a-09656e-openshift-default-6d5c46dfc from 0 to 1 26m Normal AddedLabel gateway/e2e-ipp-a-09656e Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-09656e 26m Normal AddedInterface pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Add eth0 [10.134.0.178/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-09656e-54fcb9c5db Error creating: pods "payload-processing-e2e-ipp-a-09656e-54fcb9c5db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-09656e: serviceaccount "payload-processing-e2e-ipp-a-09656e" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c Created pod: payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-09656e Scaled up replica set payload-processing-e2e-ipp-a-09656e-54fcb9c5db from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-09656e Scaled up replica set payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c Error creating: pods "payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-09656e: serviceaccount "payload-processing-e2e-ipp-a-09656e" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-09656e-54fcb9c5db Created pod: payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv 26m Normal SuccessfulCreate replicaset/e2e-ipp-b-829cb0-openshift-default-fd77c8645 Created pod: e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-6b4c49-5cdb7cdc9f-7448l Readiness probe failed: timeout: failed to connect service "10.134.0.147:50052" within 1s: context deadline exceeded 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv Add eth0 [10.133.0.202/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm Container started 26m Normal ScalingReplicaSet deployment/e2e-ipp-b-829cb0-openshift-default Scaled up replica set e2e-ipp-b-829cb0-openshift-default-fd77c8645 from 0 to 1 26m Normal Pulled pod/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm Add eth0 [10.133.0.201/23] from ovn-kubernetes 26m Normal AddedLabel gateway/e2e-ipp-b-829cb0 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-829cb0 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm Container created 26m Normal Started pod/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv Container started 26m Normal Created pod/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv Container created 26m Normal AddedInterface pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Add eth0 [10.132.1.65/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-829cb0 Scaled up replica set payload-processing-e2e-ipp-b-829cb0-6f767dc558 from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-829cb0 Scaled up replica set payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-829cb0-6f767dc558 Error creating: pods "payload-processing-e2e-ipp-b-829cb0-6f767dc558-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-829cb0: serviceaccount "payload-processing-e2e-ipp-b-829cb0" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47 Created pod: payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-829cb0-6f767dc558 Created pod: payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-5a355b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-5a355b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal AddedInterface pod/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 Add eth0 [10.133.0.204/23] from ovn-kubernetes 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47 Error creating: pods "payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-829cb0: serviceaccount "payload-processing-e2e-ipp-b-829cb0" not found 26m Normal Started pod/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 Container started 26m Normal Created pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Container created 26m Normal Pulled pod/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Started pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Container started 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Add eth0 [10.134.0.179/23] from ovn-kubernetes 26m Normal Created pod/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 Container created 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-e25317-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-e25317-openshift-default Scaled up replica set e2e-ipp-a-e25317-openshift-default-58f4485c6f from 1 to 4 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts 26m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Add eth0 [10.133.0.205/23] from ovn-kubernetes 26m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Add eth0 [10.132.1.66/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Container created 26m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Container started 26m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Container started 26m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Add eth0 [10.132.1.67/23] from ovn-kubernetes 26m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Container created 26m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Container created 26m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Container started 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-ad047b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Container created 26m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Container started 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-ad047b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Normal AddedLabel gateway/e2e-ipp-a-0bb375 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-0bb375 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-0bb375-openshift-default Scaled up replica set e2e-ipp-a-0bb375-openshift-default-6689c9665 from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-0bb375-openshift-default-6689c9665 Created pod: e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq 26m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Startup probe error: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused... 26m Normal AddedInterface pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Add eth0 [10.134.0.180/23] from ovn-kubernetes 26m Normal Started pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Container started 26m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Startup probe failed: Get "http://10.134.0.173:15021/healthz/ready": dial tcp 10.134.0.173:15021: connect: connection refused 26m Normal Pulled pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Startup probe error: Get "http://10.133.0.205:15021/healthz/ready": dial tcp 10.133.0.205:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Startup probe failed: Get "http://10.133.0.205:15021/healthz/ready": dial tcp 10.133.0.205:15021: connect: connection refused 26m Normal Created pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Container created 26m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Startup probe error: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Startup probe failed: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Startup probe error: Get "http://10.132.1.66:15021/healthz/ready": dial tcp 10.132.1.66:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Startup probe failed: Get "http://10.132.1.66:15021/healthz/ready": dial tcp 10.132.1.66:15021: connect: connection refused 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-0bb375 Scaled up replica set payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff from 0 to 1 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-0bb375 Scaled up replica set payload-processing-e2e-ipp-a-0bb375-5bb6579594 from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff Error creating: pods "payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-0bb375: serviceaccount "payload-processing-e2e-ipp-a-0bb375" not found 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff Created pod: payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-0bb375-5bb6579594 Error creating: pods "payload-processing-e2e-ipp-a-0bb375-5bb6579594-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-0bb375: serviceaccount "payload-processing-e2e-ipp-a-0bb375" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-0bb375-5bb6579594 Created pod: payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Created pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Container created 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Add eth0 [10.134.0.182/23] from ovn-kubernetes 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Add eth0 [10.134.0.181/23] from ovn-kubernetes 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Container created 26m Normal Started pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Container started 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e25317-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e25317-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/payload-processing-e2e-ipp-b-ad047b-678ff75974-mh7cq Stopping container payload-processing 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-ad047b-54f599749f-5z9zm Stopping container payload-pre-processing 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e25317-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e25317-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-09656e-79bd965b6c-q6fdm Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-ipp-a-09656e-54fcb9c5db-4qctv Stopping container payload-processing 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-ad047b-66fdf5c576 Created pod: payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9 Error creating: pods "payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ad047b: serviceaccount "payload-processing-e2e-ipp-b-ad047b" not found 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-ad047b Scaled up replica set payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-ad047b Scaled up replica set payload-processing-e2e-ipp-b-ad047b-66fdf5c576 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-ad047b-66fdf5c576 Error creating: pods "payload-processing-e2e-ipp-b-ad047b-66fdf5c576-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-ad047b: serviceaccount "payload-processing-e2e-ipp-b-ad047b" not found 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 Add eth0 [10.133.0.207/23] from ovn-kubernetes 25m Normal Killing pod/payload-processing-e2e-ipp-b-829cb0-6f767dc558-nxb98 Stopping container payload-processing 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9 Created pod: payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd 25m Normal Pulled pod/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Killing pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Stopping container payload-pre-processing 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd Container created 25m Normal Created pod/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd Add eth0 [10.134.0.183/23] from ovn-kubernetes 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-e25317-openshift-default New size: 7; reason: All metrics below target 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t 25m Normal ScalingReplicaSet deployment/e2e-ipp-a-e25317-openshift-default Scaled up replica set e2e-ipp-a-e25317-openshift-default-58f4485c6f from 4 to 7 25m Normal SuccessfulCreate replicaset/e2e-ipp-a-e25317-openshift-default-58f4485c6f Created pod: e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 25m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Add eth0 [10.133.0.209/23] from ovn-kubernetes 25m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Container started 25m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Container created 25m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Add eth0 [10.133.0.208/23] from ovn-kubernetes 25m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Container started 25m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Container created 25m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Startup probe failed: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Startup probe error: Get "http://10.133.0.209:15021/healthz/ready": dial tcp 10.133.0.209:15021: connect: connection refused... 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-09656e-88b99879 Error creating: pods "payload-pre-processing-e2e-ipp-a-09656e-88b99879-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-09656e: serviceaccount "payload-processing-e2e-ipp-a-09656e" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-09656e-88b99879 Created pod: payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-09656e Scaled up replica set payload-processing-e2e-ipp-a-09656e-7bd85bc696 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-09656e Scaled up replica set payload-pre-processing-e2e-ipp-a-09656e-88b99879 from 0 to 1 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-09656e-7bd85bc696 Created pod: payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-09656e-7bd85bc696 Error creating: pods "payload-processing-e2e-ipp-a-09656e-7bd85bc696-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-09656e: serviceaccount "payload-processing-e2e-ipp-a-09656e" not found 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 Add eth0 [10.133.0.210/23] from ovn-kubernetes 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-1783e3-openshift-default Scaled down replica set e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc from 9 to 8 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default New size: 8; reason: All metrics below target 25m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc Deleted pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-jdxd4 25m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Startup probe failed: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused 25m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Startup probe error: Get "http://10.133.0.208:15021/healthz/ready": dial tcp 10.133.0.208:15021: connect: connection refused... 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-829cb0 Scaled up replica set payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-829cb0 Scaled up replica set payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd Error creating: pods "payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-829cb0: serviceaccount "payload-processing-e2e-ipp-b-829cb0" not found 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c Error creating: pods "payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-829cb0: serviceaccount "payload-processing-e2e-ipp-b-829cb0" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd Created pod: payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c Created pod: payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd Add eth0 [10.133.0.211/23] from ovn-kubernetes 25m Normal Created pod/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg Container created 25m Normal Started pod/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg Container started 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg Add eth0 [10.133.0.212/23] from ovn-kubernetes 25m Normal Pulled pod/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd Container started 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd Container created 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Liveness probe failed: dial tcp 10.134.0.80:50052: i/o timeout 25m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Readiness probe failed: timeout: health rpc did not complete within 1s 25m Warning Unhealthy pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Readiness probe failed: dial tcp 10.134.0.177:9004: i/o timeout 25m Normal Killing pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Stopping container istio-proxy 25m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 Deleted pod: e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-d6e25f-openshift-default New size: 2; reason: All metrics below target 25m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-d6e25f-openshift-default Scaled down replica set e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47 from 3 to 2 25m Warning ProbeError pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-s6f5j Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59 Error creating: pods "payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-0bb375: serviceaccount "payload-processing-e2e-ipp-a-0bb375" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59 Created pod: payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-0bb375 Scaled up replica set payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59 from 0 to 1 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-0bb375 Scaled up replica set payload-processing-e2e-ipp-a-0bb375-76979cf7dd from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-0bb375-76979cf7dd Error creating: pods "payload-processing-e2e-ipp-a-0bb375-76979cf7dd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-0bb375: serviceaccount "payload-processing-e2e-ipp-a-0bb375" not found 25m Normal Pulled pod/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Started pod/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c Container started 25m Normal Created pod/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf Container created 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-0bb375-76979cf7dd Created pod: payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf 25m Normal Created pod/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c Container created 25m Normal Started pod/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c Add eth0 [10.133.0.213/23] from ovn-kubernetes 25m Normal AddedInterface pod/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf Add eth0 [10.133.0.214/23] from ovn-kubernetes 24m Normal Started pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Container started 24m Normal Created pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Container created 24m Normal Pulled pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning Unhealthy pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Startup probe failed: Get "http://10.132.1.65:15021/healthz/ready": dial tcp 10.132.1.65:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Startup probe error: Get "http://10.132.1.65:15021/healthz/ready": dial tcp 10.132.1.65:15021: connect: connection refused... 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc Created pod: e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-a00c6a-openshift-default Scaled up replica set e2e-ipp-b-a00c6a-openshift-default-7f74497cbc from 0 to 1 24m Normal AddedLabel gateway/e2e-ipp-b-a00c6a Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-a00c6a 24m Normal AddedInterface pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Add eth0 [10.132.1.73/23] from ovn-kubernetes 24m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Readiness probe failed: timeout: failed to connect service "10.134.0.85:50052" within 1s: context deadline exceeded 24m Warning ProbeError pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Readiness probe error: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Readiness probe failed: Get "http://10.134.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Readiness probe error: Get "http://10.134.0.166:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning ProbeError pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Readiness probe error: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Readiness probe failed: Get "http://10.134.0.155:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Readiness probe error: Get "http://10.134.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Readiness probe failed: Get "http://10.134.0.163:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Readiness probe error: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Readiness probe failed: Get "http://10.134.0.161:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Readiness probe failed: Get "http://10.134.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Readiness probe error: Get "http://10.134.0.164:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning ProbeError pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Readiness probe error: Get "http://10.134.0.159:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Readiness probe failed: timeout: failed to connect service "10.134.0.99:50052" within 1s: context deadline exceeded 24m Warning Unhealthy pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Readiness probe failed: Get "http://10.134.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Readiness probe error: Get "http://10.134.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Readiness probe failed: timeout: failed to connect service "10.134.0.160:50052" within 1s: context deadline exceeded 24m Warning Unhealthy pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Readiness probe failed: Get "http://10.134.0.164:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Readiness probe error: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Readiness probe error: Get "http://10.134.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Readiness probe failed: Get "http://10.134.0.173:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Readiness probe failed: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Readiness probe failed: timeout: failed to connect service "10.134.0.83:50052" within 1s: context deadline exceeded 24m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Readiness probe error: Get "http://10.134.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Readiness probe failed: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Readiness probe error: Get "http://10.134.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Readiness probe failed: Get "http://10.134.0.171:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Readiness probe error: Get "http://10.134.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Readiness probe failed: Get "http://10.134.0.165:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Readiness probe error: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Readiness probe failed: Get "http://10.134.0.162:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Readiness probe error: Get "http://10.134.0.61:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Readiness probe failed: Get "http://10.134.0.68:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Readiness probe failed: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Readiness probe failed: timeout: failed to connect service "10.134.0.80:50052" within 1s: context deadline exceeded 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Readiness probe failed: timeout: failed to connect service "10.134.0.167:50052" within 1s: context deadline exceeded 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a00c6a Scaled up replica set payload-processing-e2e-ipp-b-a00c6a-668fb87544 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a00c6a-668fb87544 Created pod: payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f Created pod: payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx 24m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Readiness probe failed: timeout: failed to connect service "10.134.0.168:50052" within 1s: context deadline exceeded 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a00c6a-668fb87544 Error creating: pods "payload-processing-e2e-ipp-b-a00c6a-668fb87544-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a00c6a: serviceaccount "payload-processing-e2e-ipp-b-a00c6a" not found 24m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Readiness probe failed: Get "http://10.134.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Readiness probe failed: timeout: failed to connect service "10.134.0.182:50052" within 1s: context deadline exceeded 24m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Readiness probe failed: timeout: failed to connect service "10.134.0.86:50052" within 1s: context deadline exceeded 24m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-mrlcb Readiness probe error: Get "http://10.134.0.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Readiness probe failed: timeout: failed to connect service "10.134.0.156:50052" within 1s: context deadline exceeded 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f Error creating: pods "payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a00c6a: serviceaccount "payload-processing-e2e-ipp-b-a00c6a" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a00c6a Scaled up replica set payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f from 0 to 1 24m Warning Unhealthy pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Readiness probe failed: timeout: failed to connect service "10.134.0.181:50052" within 1s: context deadline exceeded 24m Normal Created pod/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b Container created 24m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr Readiness probe failed: timeout: failed to connect service "10.134.0.179:50052" within 1s: context deadline exceeded 24m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Readiness probe failed: timeout: failed to connect service "10.134.0.157:50052" within 1s: context deadline exceeded 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx Add eth0 [10.133.0.215/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b Container started 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx Container started 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b Add eth0 [10.133.0.216/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Pulled pod/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx Container created 24m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Readiness probe error: Get "http://10.134.0.62:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Readiness probe failed: Get "http://10.134.0.62:8080/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-829cb0-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-829cb0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 24m Warning FailedKillPod pod/payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr error killing pod: failed to "KillPodSandbox" for "6d07a690-ca3c-4d37-941d-68b7ef26f366" with KillPodSandboxError: "rpc error: code = Unknown desc = network teardown failed for pod sandbox k8s_payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr_openshift-ingress_6d07a690-ca3c-4d37-941d-68b7ef26f366_0(abcde0a3dff5205679f3896b660d1268299e3cdc5c6667ed4fca6ed225057570): error removing pod openshift-ingress_payload-pre-processing-e2e-ipp-b-829cb0-8f6cc8d47-f44nr from CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (delete): netplugin failed with no error message: signal: killed" 24m Normal Killing pod/payload-processing-e2e-ipp-a-0bb375-5bb6579594-5qbkv Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-0bb375-5f666dc5ff-gh7q2 Stopping container payload-pre-processing 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal AddedInterface pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Add eth0 [10.134.0.184/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n Add eth0 [10.134.0.185/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Readiness probe failed: Get "http://10.134.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Warning ProbeError pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Readiness probe error: Get "http://10.134.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Warning Unhealthy pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Readiness probe failed: Get "http://10.134.0.178:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 24m Normal Created pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Container started 24m Warning ProbeError pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Readiness probe error: Get "http://10.134.0.178:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 24m Normal Started pod/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n Container started 24m Normal Created pod/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n Container created 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-f758d9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-f758d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning ProbeError pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Startup probe error: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Startup probe failed: Get "http://10.134.0.184:15021/healthz/ready": dial tcp 10.134.0.184:15021: connect: connection refused 24m Normal AddedLabel gateway/e2e-ipp-a-2a68ac Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-2a68ac 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f Created pod: e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-2a68ac-openshift-default Scaled up replica set e2e-ipp-a-2a68ac-openshift-default-56bd457b8f from 0 to 1 24m Normal AddedInterface pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Add eth0 [10.134.0.186/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Container started 24m Normal Created pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Container created 24m Normal Started pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Container started 24m Normal Pulled pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Created pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Container created 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f Created pod: payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f Error creating: pods "payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2a68ac: serviceaccount "payload-processing-e2e-ipp-a-2a68ac" not found 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-56452a-openshift-default-f48656c47 Created pod: e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-56452a-openshift-default Scaled up replica set e2e-ipp-b-56452a-openshift-default-f48656c47 from 0 to 1 24m Normal AddedLabel gateway/e2e-ipp-b-56452a Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-56452a 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-2a68ac Scaled up replica set payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f from 0 to 1 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db Error creating: pods "payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2a68ac: serviceaccount "payload-processing-e2e-ipp-a-2a68ac" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db Created pod: payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-2a68ac Scaled up replica set payload-processing-e2e-ipp-a-2a68ac-6cb788b5db from 0 to 1 24m Normal Started pod/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 Container started 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw Container started 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw Add eth0 [10.133.0.217/23] from ovn-kubernetes 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 Add eth0 [10.134.0.187/23] from ovn-kubernetes 24m Normal AddedInterface pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Add eth0 [10.132.1.79/23] from ovn-kubernetes 24m Warning Unhealthy pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Startup probe failed: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Startup probe error: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Startup probe failed: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Startup probe error: Get "http://10.134.0.178:15021/healthz/ready": dial tcp 10.134.0.178:15021: connect: connection refused... 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-0bb375-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-0bb375-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-829cb0-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-56452a Scaled up replica set payload-processing-e2e-ipp-b-56452a-6b677f4848 from 0 to 1 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-829cb0-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-56452a-6b677f4848 Created pod: payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-56452a-6b677f4848 Error creating: pods "payload-processing-e2e-ipp-b-56452a-6b677f4848-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-56452a: serviceaccount "payload-processing-e2e-ipp-b-56452a" not found 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 Add eth0 [10.133.0.218/23] from ovn-kubernetes 24m Normal Started pod/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl Container started 24m Normal Created pod/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl Container created 24m Normal Pulled pod/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675 Created pod: payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675 Error creating: pods "payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-56452a: serviceaccount "payload-processing-e2e-ipp-b-56452a" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-56452a Scaled up replica set payload-pre-processing-e2e-ipp-b-56452a-95b5c7675 from 0 to 1 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 Container started 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 Container created 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl Add eth0 [10.134.0.188/23] from ovn-kubernetes 24m Normal Killing pod/payload-processing-e2e-ipp-a-e25317-7bcfcd9d96-hfxtb Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-5a355b-5cbb745987-tkhnl Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-ipp-a-5a355b-596cd847f6-h4xk2 Stopping container payload-processing 24m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-5a355b Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-5a355b: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-5a355b": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-5a355b, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: b03f4b3c-3b59-41cd-b785-66730f5ccd87, UID in object meta: 24m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e25317-5dd46f7c75-wbk4m Stopping container payload-pre-processing 24m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-e25317 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-e25317: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-e25317": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-e25317, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 45681008-e766-410c-8a77-7be5c2b345eb, UID in object meta: 24m Normal Killing pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-72cxz Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-tj44t Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-h7rfd Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-vpfts Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-t55x8 Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-nt4hl Stopping container istio-proxy 24m Normal Killing pod/e2e-ipp-a-e25317-openshift-default-58f4485c6f-pffq4 Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-a-5a355b-openshift-default-8576bb9c45-9c7tp Readiness probe failed: HTTP probe failed with statuscode: 503 23m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-dcjl7 23m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 23m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Container created 23m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Container started 23m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Add eth0 [10.133.0.219/23] from ovn-kubernetes 23m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a00c6a-57fc7fcd7f-rj7jx Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-a00c6a-668fb87544-crv7b Stopping container payload-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-829cb0-6b7f95ffdd-qh4xg Stopping container payload-processing 23m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-b-829cb0 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-b-829cb0: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-b-829cb0": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-b-829cb0, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6dd2324d-d9ae-4edb-a00b-55a00c80f7c0, UID in object meta: 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-829cb0-9fc54866c-hgmhd Stopping container payload-pre-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-ad047b-646fb5b6c9-tj8wd Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-ad047b-66fdf5c576-2zw75 Stopping container payload-processing 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-ad047b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-ad047b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Normal Killing pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Stopping container istio-proxy 23m Normal Killing pod/e2e-ipp-b-ad047b-openshift-default-76dc48b85-cssws Stopping container istio-proxy 23m Warning ProbeError pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-ipp-b-829cb0-openshift-default-fd77c8645-xkkb6 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd Error creating: pods "payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a00c6a: serviceaccount "payload-processing-e2e-ipp-b-a00c6a" not found 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf Error creating: pods "payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-a00c6a: serviceaccount "payload-processing-e2e-ipp-b-a00c6a" not found 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf Created pod: payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-a00c6a Scaled up replica set payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-a00c6a Scaled up replica set payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd Created pod: payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-2a68ac-85b847957f-k5xnw Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-2a68ac-6cb788b5db-87m56 Stopping container payload-processing 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh Container started 23m Normal Pulled pod/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m Container created 23m Normal Started pod/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh Add eth0 [10.133.0.220/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh Container created 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m Add eth0 [10.134.0.189/23] from ovn-kubernetes 23m Normal Killing pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Stopping container payload-pre-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-56452a-95b5c7675-td2g6 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-56452a-6b677f4848-bbhnl Stopping container payload-processing 23m Normal Pulled pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Created pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Container created 23m Normal Started pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Container started 23m Normal Killing pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-85wd6 Stopping container istio-proxy 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d Created pod: payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-2a68ac Scaled up replica set payload-processing-e2e-ipp-a-2a68ac-776d8bc74d from 0 to 1 23m Normal Created pod/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh Container created 23m Normal Started pod/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh Container started 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh Add eth0 [10.134.0.191/23] from ovn-kubernetes 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-2a68ac Scaled up replica set payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55 Created pod: payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55 Error creating: pods "payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2a68ac: serviceaccount "payload-processing-e2e-ipp-a-2a68ac" not found 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d Error creating: pods "payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-2a68ac: serviceaccount "payload-processing-e2e-ipp-a-2a68ac" not found 23m Normal Pulled pod/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr Container started 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr Container created 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr Add eth0 [10.134.0.190/23] from ovn-kubernetes 23m Warning Unhealthy pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Startup probe failed: Get "http://10.132.1.73:15021/healthz/ready": dial tcp 10.132.1.73:15021: connect: connection refused 23m Warning ProbeError pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Startup probe error: Get "http://10.132.1.73:15021/healthz/ready": dial tcp 10.132.1.73:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning ProbeError pod/e2e-shared-a-w6-d6e25f-openshift-default-5b4b4fdb47-vzdq4 Readiness probe error: HTTP probe failed with statuscode: 503... 23m Normal Killing pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Stopping container istio-proxy 23m Normal Killing pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-ktqqj Stopping container istio-proxy 23m Warning ProbeError pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Readiness probe error: HTTP probe failed with statuscode: 503... 23m Warning Unhealthy pod/e2e-shared-a-w2-702002-openshift-default-5588c5c5d9-s9gmt Readiness probe failed: HTTP probe failed with statuscode: 503 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w2-702002-768fc75ff4-pgbkt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-56452a Scaled up replica set payload-processing-e2e-ipp-b-56452a-649448cb74 from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-56452a-649448cb74 Error creating: pods "payload-processing-e2e-ipp-b-56452a-649448cb74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-56452a: serviceaccount "payload-processing-e2e-ipp-b-56452a" not found 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9 Error creating: pods "payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-56452a: serviceaccount "payload-processing-e2e-ipp-b-56452a" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-56452a-649448cb74 Created pod: payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9 Created pod: payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-56452a Scaled up replica set payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9 from 0 to 1 23m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p Container created 23m Normal AddedInterface pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Add eth0 [10.132.1.94/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567 Created pod: e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p Container started 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr Add eth0 [10.134.0.193/23] from ovn-kubernetes 23m Normal Pulled pod/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p Add eth0 [10.134.0.192/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Pulled pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-d2832c-openshift-default Scaled up replica set e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567 from 1 to 2 23m Normal Started pod/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr Container started 23m Normal Created pod/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr Container created 23m Normal Started pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Container started 23m Normal Killing pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Stopping container payload-processing 23m Normal Created pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Container created 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Stopping container payload-pre-processing 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a00c6a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a00c6a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-d6e25f-689b6f998c-mlz85 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-d6e25f-dc644b8-wmqhn Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w2-702002-594459b8b7-4rjc2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Stopping container payload-pre-processing 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-1783e3-openshift-default Scaled up replica set e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc from 8 to 10 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 22m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Add eth0 [10.132.1.96/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd 22m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Add eth0 [10.133.0.222/23] from ovn-kubernetes 22m Normal AddedInterface pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Add eth0 [10.133.0.221/23] from ovn-kubernetes 22m Normal Pulled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Add eth0 [10.134.0.194/23] from ovn-kubernetes 22m Normal AddedInterface pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Add eth0 [10.133.0.223/23] from ovn-kubernetes 22m Normal Killing pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Stopping container payload-processing 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 22m Normal Created pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Container created 22m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-521734-openshift-default Scaled up replica set e2e-shared-a-w3-521734-openshift-default-756d487fd9 from 8 to 10 22m Normal Pulled pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Pulled pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Warning ProbeError pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Startup probe error: Get "http://10.132.1.94:15021/healthz/ready": dial tcp 10.132.1.94:15021: connect: connection refused... 22m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-e2bf63-openshift-default Scaled up replica set e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 from 2 to 4 22m Warning Unhealthy pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Startup probe failed: Get "http://10.132.1.94:15021/healthz/ready": dial tcp 10.132.1.94:15021: connect: connection refused 22m Normal AddedInterface pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Add eth0 [10.134.0.195/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 Created pod: e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-521734-openshift-default-756d487fd9 (combined from similar events): Created pod: e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk 22m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc (combined from similar events): Created pod: e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t 22m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Container created 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-e2bf63-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 22m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89 Created pod: e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw 22m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Container started 22m Normal AddedLabel gateway/e2e-ipp-b-db715b Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-db715b 22m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Container created 22m Normal Created pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Container created 22m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Container started 22m Normal Started pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Container started 22m Normal Pulled pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Container created 22m Normal Started pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Container started 22m Normal Created pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Container created 22m Normal Started pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Container started 22m Normal Started pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Container started 22m Normal ScalingReplicaSet deployment/e2e-ipp-b-db715b-openshift-default Scaled up replica set e2e-ipp-b-db715b-openshift-default-d756b6495 from 0 to 1 22m Normal SuccessfulCreate replicaset/e2e-ipp-b-db715b-openshift-default-d756b6495 Created pod: e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb 22m Normal AddedInterface pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Add eth0 [10.132.1.97/23] from ovn-kubernetes 22m Warning Unhealthy pod/payload-processing-e2e-shared-b-w2-de0cba-7dc764dbd4-glwvs Readiness probe failed: timeout: failed to connect service "10.134.0.105:50052" within 1s: context deadline exceeded 22m Normal Killing pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-jsm5s Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Startup probe failed: Get "http://10.132.1.96:15021/healthz/ready": dial tcp 10.132.1.96:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Startup probe error: Get "http://10.132.1.96:15021/healthz/ready": dial tcp 10.132.1.96:15021: connect: connection refused... 22m Normal Killing pod/e2e-shared-b-w6-da34ca-openshift-default-6db76b45c-g9k8c Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Startup probe error: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Startup probe failed: Get "http://10.134.0.195:15021/healthz/ready": dial tcp 10.134.0.195:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 22m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Startup probe error: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused... 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7 Created pod: payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 22m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Startup probe failed: Get "http://10.133.0.221:15021/healthz/ready": dial tcp 10.133.0.221:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Startup probe error: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused... 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-db715b-769b6dff6d Created pod: payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 Add eth0 [10.134.0.196/23] from ovn-kubernetes 22m Warning Unhealthy pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Startup probe failed: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused 22m Warning ProbeError pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Startup probe error: Get "http://10.133.0.222:15021/healthz/ready": dial tcp 10.133.0.222:15021: connect: connection refused... 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7 Error creating: pods "payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-db715b: serviceaccount "payload-processing-e2e-ipp-b-db715b" not found 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-db715b Scaled up replica set payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7 from 0 to 1 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-db715b Scaled up replica set payload-processing-e2e-ipp-b-db715b-769b6dff6d from 0 to 1 22m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Startup probe failed: Get "http://10.133.0.223:15021/healthz/ready": dial tcp 10.133.0.223:15021: connect: connection refused 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-db715b-769b6dff6d Error creating: pods "payload-processing-e2e-ipp-b-db715b-769b6dff6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-db715b: serviceaccount "payload-processing-e2e-ipp-b-db715b" not found 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Add eth0 [10.133.0.224/23] from ovn-kubernetes 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 Container started 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Container created 22m Normal Started pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Container started 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a00c6a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a00c6a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-0bb375 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-0bb375: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-0bb375": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-0bb375, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 55566057-0d75-4c21-9bad-c6b11fcf2177, UID in object meta: 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-0bb375-6446dc7b59-bpd2c Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-0bb375-76979cf7dd-vvvpf Stopping container payload-processing 22m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-da34ca-5d7fb98df4-m5gv8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-da34ca-c997c8c67-7zfp5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Killing pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Stopping container payload-pre-processing 22m Normal Killing pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Stopping container istio-proxy 22m Warning ProbeError pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Readiness probe error: Get "http://10.134.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 22m Warning Unhealthy pod/e2e-ipp-a-0bb375-openshift-default-6689c9665-mjzkq Readiness probe failed: Get "http://10.134.0.180:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 22m Normal Killing pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-f7ckq Stopping container istio-proxy 22m Normal Killing pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Stopping container istio-proxy 22m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w2-de0cba-78f5f7494c-fhbbl Readiness probe failed: timeout: failed to connect service "10.134.0.106:50052" within 1s: context deadline exceeded 22m Warning ProbeError pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-shared-b-w2-de0cba-openshift-default-7788b4bcd7-whmj5 Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal AddedLabel gateway/e2e-ipp-a-c6bb56 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-c6bb56 22m Normal ScalingReplicaSet deployment/e2e-ipp-a-c6bb56-openshift-default Scaled up replica set e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9 from 0 to 1 22m Normal SuccessfulCreate replicaset/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9 Created pod: e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 22m Normal AddedInterface pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Add eth0 [10.133.0.226/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8 Error creating: pods "payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6bb56: serviceaccount "payload-processing-e2e-ipp-a-c6bb56" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c6bb56 Scaled up replica set payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8 Created pod: payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c6bb56 Scaled up replica set payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499 from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499 Created pod: payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499 Error creating: pods "payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6bb56: serviceaccount "payload-processing-e2e-ipp-a-c6bb56" not found 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn Add eth0 [10.133.0.227/23] from ovn-kubernetes 22m Normal Created pod/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn Container started 22m Normal Started pod/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng Container started 22m Normal Created pod/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng Container created 22m Normal Pulled pod/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng Add eth0 [10.133.0.228/23] from ovn-kubernetes 22m Normal Killing pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Stopping container payload-processing 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-fb9597-b5958d4dc-2t46h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Normal Pulled pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Container created 22m Normal Started pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Container started 22m Normal Started pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Container started 22m Normal Created pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Container created 22m Normal Pulled pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-56452a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-56452a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning Unhealthy pod/payload-processing-e2e-worker-w6-fb9597-86489d9d58-t5b2n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning ProbeError pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Startup probe error: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Startup probe failed: Get "http://10.134.0.186:15021/healthz/ready": dial tcp 10.134.0.186:15021: connect: connection refused 22m Warning ProbeError pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Startup probe error: Get "http://10.132.1.79:15021/healthz/ready": dial tcp 10.132.1.79:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Startup probe failed: Get "http://10.132.1.79:15021/healthz/ready": dial tcp 10.132.1.79:15021: connect: connection refused 22m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 22m Normal ScalingReplicaSet deployment/e2e-worker-w6-fb9597-openshift-default Scaled up replica set e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 from 2 to 3 22m Normal SuccessfulCreate replicaset/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7 Created pod: e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p 22m Normal AddedInterface pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Add eth0 [10.132.1.106/23] from ovn-kubernetes 22m Normal Created pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Container created 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-db715b-67d594b7d7-fv9b5 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Stopping container payload-processing 22m Normal Started pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Container started 22m Normal Pulled pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Killing pod/payload-processing-e2e-ipp-b-a00c6a-6b5b58b4cd-24z4m Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-a00c6a-66fcc5cdcf-zstfh Stopping container payload-pre-processing 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-2a68ac-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-2a68ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning Unhealthy pod/payload-processing-e2e-worker-w2-f033b1-7f46dbb79b-t2ptt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-ipp-b-db715b-769b6dff6d-zrq58 Readiness probe failed: timeout: failed to connect service "10.133.0.224:50052" within 1s: context deadline exceeded 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-f033b1-54b598887f-rmgb9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Startup probe error: Get "http://10.132.1.106:15021/healthz/ready": dial tcp 10.132.1.106:15021: connect: connection refused... 22m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Startup probe failed: Get "http://10.132.1.106:15021/healthz/ready": dial tcp 10.132.1.106:15021: connect: connection refused 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979 Created pod: payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-db715b-7c7d74974 Error creating: pods "payload-processing-e2e-ipp-b-db715b-7c7d74974-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-db715b: serviceaccount "payload-processing-e2e-ipp-b-db715b" not found 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-db715b Scaled up replica set payload-processing-e2e-ipp-b-db715b-7c7d74974 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-db715b-7c7d74974 Created pod: payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979 Error creating: pods "payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-db715b: serviceaccount "payload-processing-e2e-ipp-b-db715b" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-db715b Scaled up replica set payload-pre-processing-e2e-ipp-b-db715b-794ccf4979 from 0 to 1 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 Container created 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 Add eth0 [10.133.0.232/23] from ovn-kubernetes 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg Container started 21m Normal Created pod/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg Container created 21m Normal Pulled pod/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg Add eth0 [10.133.0.231/23] from ovn-kubernetes 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 Container started 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c6bb56-64d4f96499-xqzfn Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-a-c6bb56-bdc7d9dd8-8b2ng Stopping container payload-processing 21m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-d2832c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-56452a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-56452a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-a00c6a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-a00c6a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c6bb56-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c6bb56-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-2c5135-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-2c5135-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-fb9597-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-e3890e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-1783e3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-f033b1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-1f3a6b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-b0b7a3-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-0def36-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-f758d9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-e2bf63-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-e2bf63-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-53b784-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-f758d9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-521734-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-51e3b7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-ff3e70-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-db715b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-db715b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-c6bb56 Scaled up replica set payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55 from 0 to 1 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-c6bb56 Scaled up replica set payload-processing-e2e-ipp-a-c6bb56-749c9ddc86 from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86 Created pod: payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86 Error creating: pods "payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6bb56: serviceaccount "payload-processing-e2e-ipp-a-c6bb56" not found 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55 Error creating: pods "payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-c6bb56: serviceaccount "payload-processing-e2e-ipp-a-c6bb56" not found 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-2a68ac-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-2a68ac-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server is currently unable to handle the request (get pods.metrics.k8s.io) 21m Normal Started pod/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 Container started 21m Normal Pulled pod/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 Container created 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55 Created pod: payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 Add eth0 [10.133.0.233/23] from ovn-kubernetes 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn Container started 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn Add eth0 [10.133.0.234/23] from ovn-kubernetes 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn Container created 21m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-9p2wr Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-l7wcc Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-rfvbl Stopping container istio-proxy 21m Normal Killing pod/e2e-worker-w6-fb9597-openshift-default-749f7c6fc7-pmk2p Stopping container istio-proxy 21m Warning ProbeError pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-worker-w2-f033b1-openshift-default-5b679ffdf9-pt5wm Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Created pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Container created 21m Normal Started pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Container started 21m Warning Unhealthy pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Readiness probe failed: Get "http://10.133.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 21m Warning ProbeError pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Readiness probe error: Get "http://10.133.0.226:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 21m Normal Pulled pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-db715b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-db715b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Normal Pulled pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Started pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Container started 21m Normal Created pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Container created 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-2a68ac-b78587d55-zjsdr Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-a-2a68ac-776d8bc74d-z7jxh Stopping container payload-processing 21m Normal Killing pod/e2e-ipp-b-a00c6a-openshift-default-7f74497cbc-9fl6f Stopping container istio-proxy 21m Warning ProbeError pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Startup probe error: Get "http://10.132.1.97:15021/healthz/ready": dial tcp 10.132.1.97:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Startup probe failed: Get "http://10.132.1.97:15021/healthz/ready": dial tcp 10.132.1.97:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Startup probe error: Get "http://10.133.0.226:15021/healthz/ready": dial tcp 10.133.0.226:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Startup probe failed: Get "http://10.133.0.226:15021/healthz/ready": dial tcp 10.133.0.226:15021: connect: connection refused 21m Normal AddedInterface pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Add eth0 [10.133.0.235/23] from ovn-kubernetes 21m Normal AddedLabel gateway/e2e-ipp-b-60807f Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-60807f 21m Normal ScalingReplicaSet deployment/e2e-ipp-b-60807f-openshift-default Scaled up replica set e2e-ipp-b-60807f-openshift-default-7b76b969c9 from 0 to 1 21m Normal SuccessfulCreate replicaset/e2e-ipp-b-60807f-openshift-default-7b76b969c9 Created pod: e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9 Error creating: pods "payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-60807f: serviceaccount "payload-processing-e2e-ipp-b-60807f" not found 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c6bb56-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-60807f-cfdc57d6d Error creating: pods "payload-processing-e2e-ipp-b-60807f-cfdc57d6d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-60807f: serviceaccount "payload-processing-e2e-ipp-b-60807f" not found 21m Normal Killing pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-60807f Scaled up replica set payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9 from 0 to 1 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c6bb56-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-60807f Scaled up replica set payload-processing-e2e-ipp-b-60807f-cfdc57d6d from 0 to 1 21m Normal Created pod/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 Container created 21m Normal Pulled pod/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9 Created pod: payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-60807f-cfdc57d6d Created pod: payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 21m Warning ProbeError pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf Add eth0 [10.134.0.197/23] from ovn-kubernetes 21m Normal AddedInterface pod/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 Add eth0 [10.134.0.198/23] from ovn-kubernetes 21m Normal Started pod/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 Container started 21m Warning Unhealthy pod/e2e-ipp-a-2a68ac-openshift-default-56bd457b8f-tsq9n Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Started pod/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf Container started 21m Normal Created pod/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf Container created 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Stopping container payload-processing 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-c6bb56-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-c6bb56-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-db715b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-db715b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Normal Killing pod/payload-processing-e2e-ipp-b-db715b-7c7d74974-t8wsg Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-db715b-794ccf4979-6bdp8 Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-e3890e-7898d88d67-jdf6j Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-e3890e-8689f59474-s8blm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-fg4gq Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-9888g Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-grcn9 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w1-e3890e-openshift-default-6975b574b5-8gm5l Stopping container istio-proxy 20m Normal Killing pod/payload-processing-e2e-ipp-b-60807f-cfdc57d6d-ns5d4 Stopping container payload-processing 20m Normal Killing pod/e2e-ipp-b-db715b-openshift-default-d756b6495-dntjb Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-60807f-7d8857bcf9-r87wf Stopping container payload-pre-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Stopping container payload-processing 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-60807f Scaled up replica set payload-processing-e2e-ipp-b-60807f-7674544b74 from 0 to 1 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-60807f-7674544b74 Error creating: pods "payload-processing-e2e-ipp-b-60807f-7674544b74-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-60807f: serviceaccount "payload-processing-e2e-ipp-b-60807f" not found 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-60807f Scaled up replica set payload-pre-processing-e2e-ipp-b-60807f-cb55749f from 0 to 1 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-60807f-cb55749f Created pod: payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-60807f-7674544b74 Created pod: payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 Add eth0 [10.134.0.203/23] from ovn-kubernetes 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-60807f-cb55749f Error creating: pods "payload-pre-processing-e2e-ipp-b-60807f-cb55749f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-60807f: serviceaccount "payload-processing-e2e-ipp-b-60807f" not found 20m Normal Started pod/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 Container started 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh Add eth0 [10.133.0.242/23] from ovn-kubernetes 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Created pod/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh Container started 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh Container created 20m Normal Killing pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-t5rxj Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning ProbeError pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w1-51e3b7-openshift-default-b5dc5b5cc-bt485 Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Created pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Container created 20m Normal Pulled pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Container started 20m Normal Killing pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-worker-w1-53b784-566d7f64fd-8vbvl Stopping container payload-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-51e3b7-bbcf88cf-zrf8h Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-51e3b7-769bbdbc5d-7zxkm Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Stopping container istio-proxy 20m Warning ProbeError pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Startup probe error: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused... 20m Warning Unhealthy pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Startup probe failed: Get "http://10.133.0.235:15021/healthz/ready": dial tcp 10.133.0.235:15021: connect: connection refused 20m Warning ProbeError pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w5-f758d9-openshift-default-765d9d8f7f-x4zkw Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-f758d9-67b9b75799-fvfsc Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-f758d9-554fcc4d84-gg7tt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Stopping container payload-processing 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-60807f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-53b784-5db67d6c99-vzhvb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-60807f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Normal Killing pod/payload-processing-e2e-ipp-a-c6bb56-749c9ddc86-kb8p8 Stopping container payload-processing 19m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-c6bb56 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-c6bb56: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-c6bb56": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-c6bb56, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 3cf4e3e4-4742-4f9b-9dbf-bd2db345ded1, UID in object meta: 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-c6bb56-76dc475c55-r59sn Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-b-w5-2c5135-openshift-default-7d55d56c89-8zbqw Stopping container istio-proxy 19m Normal Killing pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-zk4fw Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w1-53b784-openshift-default-7d967d7c57-2m7b9 Stopping container istio-proxy 19m Warning ProbeError pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-ipp-a-c6bb56-openshift-default-54d9fcd6f9-8rfh7 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-2c5135-66556bcd99-7fs67 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-worker-w5-b0b7a3-769756b4c4-gcwr6 Stopping container payload-processing 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-2c5135-7b9d8c46c4-kmfjj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal ScalingReplicaSet deployment/e2e-ipp-a-09656e-openshift-default Scaled up replica set e2e-ipp-a-09656e-openshift-default-6d5c46dfc from 1 to 2 19m Normal Pulled pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal AddedInterface pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Add eth0 [10.133.0.249/23] from ovn-kubernetes 19m Normal Created pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Container created 19m Normal Started pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Container started 19m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-09656e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 19m Normal SuccessfulCreate replicaset/e2e-ipp-a-09656e-openshift-default-6d5c46dfc Created pod: e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-60807f-cb55749f-rddvh Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-b-60807f-7674544b74-648g7 Stopping container payload-processing 19m Warning ProbeError pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Startup probe error: Get "http://10.133.0.249:15021/healthz/ready": dial tcp 10.133.0.249:15021: connect: connection refused... 19m Warning Unhealthy pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Startup probe failed: Get "http://10.133.0.249:15021/healthz/ready": dial tcp 10.133.0.249:15021: connect: connection refused 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-09656e-88b99879-zmdd8 Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-09656e-7bd85bc696-r5c8n Stopping container payload-processing 19m Warning ProbeError pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Killing pod/e2e-ipp-b-60807f-openshift-default-7b76b969c9-xnljd Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Stopping container payload-processing 19m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-bnccn Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-b0b7a3-585889448c-vsh64 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w5-b0b7a3-openshift-default-555bd95f85-gcprv Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Stopping container payload-pre-processing 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-96xpk Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-9twt5 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-5hvsq Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-wwm27 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-l4fsd Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-gq9s2 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-nh2cj Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-v472l Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-twzg9 Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w3-521734-openshift-default-756d487fd9-h428k Stopping container istio-proxy 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-521734-f8998689f-p7gqq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-521734-867d896c58-kg8ll Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-f6hxz Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j9c9b Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gb72l Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-gnn7w Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-kq7dh Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-5nfvv Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-j7cnw Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-cv5vl Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-bzr2t Stopping container istio-proxy 19m Normal Killing pod/e2e-shared-a-w4-1783e3-openshift-default-6d6647c4cc-mq89k Stopping container istio-proxy 19m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Stopping container payload-processing 18m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-1783e3-58446fb578-ddtf2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Stopping container istio-proxy 18m Warning ProbeError pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-srtx9 Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-1783e3-6b45cff6f9-rbgv2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning ProbeError pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-b-w3-d2832c-openshift-default-6d8cc8f567-bj6sb Readiness probe failed: HTTP probe failed with statuscode: 503 18m Normal Killing pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-kvt8p Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-ztrgz Stopping container istio-proxy 18m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-rlwzv Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-shared-b-w4-e2bf63-openshift-default-7cd68f8c89-qprrw Readiness probe failed: HTTP probe failed with statuscode: 503 18m Normal Killing pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Stopping container payload-processing 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-d2832c-7584ddcdbd-8qp5x Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-d2832c-c4cc54dc5-m6f9c Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-qllnr Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-ipp-a-09656e-openshift-default-6d5c46dfc-t9gnm Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Stopping container payload-pre-processing 18m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-e2bf63-68fd867c6b-qbtk2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-e2bf63-7f476f467f-bxjl7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-56452a-9f4f946f9-6vz7p Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-b-56452a-649448cb74-vk8fr Stopping container payload-processing 18m Warning Unhealthy pod/payload-processing-e2e-worker-w4-0def36-55888856c8-zff9g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-ff3e70-549bf5bb59-tsv9r Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-processing-e2e-worker-w3-ff3e70-6bfb58bc86-8lhs8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-0def36-69fb87758c-kcs2n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-4pl9f Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-5qhkp Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w3-ff3e70-openshift-default-68794df669-hdcb4 Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Stopping container istio-proxy 18m Normal Killing pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Stopping container istio-proxy 18m Normal Killing pod/e2e-ipp-b-56452a-openshift-default-f48656c47-5z2ch Stopping container istio-proxy 18m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-rsqjp Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning Unhealthy pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-worker-w4-0def36-openshift-default-7d684697cc-vbr9g Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Stopping container payload-processing 18m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 3; reason: All metrics below target 18m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5bc884dd87 from 4 to 3 18m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5bc884dd87 Deleted pod: maas-default-gateway-openshift-default-5bc884dd87-p4lpt 18m Normal Killing pod/maas-default-gateway-openshift-default-5bc884dd87-p4lpt Stopping container istio-proxy 17m Warning Unhealthy pod/payload-processing-e2e-worker-w0-1f3a6b-686494f97b-8st9g Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-bjrqt Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w0-1f3a6b-openshift-default-db74bcc99-pcrl7 Stopping container istio-proxy 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-1f3a6b-697d96996c-df8cw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-dcjl7 Stopping container discovery 17m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-dcjl7 17m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 17m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 3 to 6 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-prd6f 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-tmn2k 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-cjw9g 17m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Container created 17m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Container created 17m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Container started 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Add eth0 [10.132.1.129/23] from ovn-kubernetes 17m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Container created 17m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Add eth0 [10.133.1.9/23] from ovn-kubernetes 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Add eth0 [10.134.0.204/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Container started 17m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Container started 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Startup probe failed: Get "http://10.133.1.9:15021/healthz/ready": dial tcp 10.133.1.9:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Startup probe error: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-prd6f Startup probe failed: Get "http://10.134.0.204:15021/healthz/ready": dial tcp 10.134.0.204:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Startup probe error: Get "http://10.132.1.129:15021/healthz/ready": dial tcp 10.132.1.129:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Startup probe failed: Get "http://10.132.1.129:15021/healthz/ready": dial tcp 10.132.1.129:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Startup probe error: Get "http://10.133.1.9:15021/healthz/ready": dial tcp 10.133.1.9:15021: connect: connection refused... 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-gwqw2 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 6 to 7 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Add eth0 [10.133.1.10/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Container started 16m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Startup probe error: Get "http://10.133.1.10:15021/healthz/ready": dial tcp 10.133.1.10:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Startup probe failed: Get "http://10.133.1.10:15021/healthz/ready": dial tcp 10.133.1.10:15021: connect: connection refused 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-8xkxn 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-fp2b7 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5bc884dd87 Created pod: maas-default-gateway-openshift-default-5bc884dd87-shr8g 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5bc884dd87 from 7 to 10 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Add eth0 [10.133.1.13/23] from ovn-kubernetes 16m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Container started 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Add eth0 [10.134.0.205/23] from ovn-kubernetes 16m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Container started 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Add eth0 [10.132.1.130/23] from ovn-kubernetes 16m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Pulled pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Created pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Container created 16m Normal Started pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Container started 15m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-dsdmp 15m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 15m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 15m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-dsdmp Stopping container discovery 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Startup probe failed: Get "http://10.132.1.130:15021/healthz/ready": dial tcp 10.132.1.130:15021: connect: connection refused 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": dial tcp 10.134.0.205:15021: connect: connection refused... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Startup probe failed: Get "http://10.133.1.13:15021/healthz/ready": dial tcp 10.133.1.13:15021: connect: connection refused 15m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-shr8g Startup probe error: Get "http://10.132.1.130:15021/healthz/ready": dial tcp 10.132.1.130:15021: connect: connection refused... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Startup probe error: Get "http://10.133.1.13:15021/healthz/ready": dial tcp 10.133.1.13:15021: connect: connection refused... 15m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Startup probe error: HTTP probe failed with statuscode: 503... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Startup probe failed: HTTP probe failed with statuscode: 503 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Startup probe failed: Get "http://10.133.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Startup probe error: Get "http://10.133.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 15m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Startup probe failed: Get "http://10.134.0.205:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 15m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Startup probe error: Get "http://10.134.0.205:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 13m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-8dg8l 13m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-8dg8l Stopping container discovery 13m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 13m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 11m Normal SuccessfulCreate replicaset/e2e-worker-single-main-096081-openshift-default-5bf85f6d49 Created pod: e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz 11m Normal AddedInterface pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Add eth0 [10.133.1.26/23] from ovn-kubernetes 11m Normal ScalingReplicaSet deployment/e2e-worker-single-main-096081-openshift-default Scaled up replica set e2e-worker-single-main-096081-openshift-default-5bf85f6d49 from 0 to 1 11m Normal AddedLabel gateway/e2e-worker-single-main-096081 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-096081 11m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-096081-74b89c6484 Created pod: payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 11m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-096081-7d87cc6fdc Created pod: payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr 11m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-096081 Scaled up replica set payload-processing-e2e-worker-single-main-096081-74b89c6484 from 0 to 1 11m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-096081-74b89c6484 Error creating: pods "payload-processing-e2e-worker-single-main-096081-74b89c6484-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-096081: serviceaccount "payload-processing-e2e-worker-single-main-096081" not found 11m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-096081-7d87cc6fdc Error creating: pods "payload-pre-processing-e2e-worker-single-main-096081-7d87cc6fdc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-096081: serviceaccount "payload-processing-e2e-worker-single-main-096081" not found 11m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-096081 Scaled up replica set payload-pre-processing-e2e-worker-single-main-096081-7d87cc6fdc from 0 to 1 11m Normal Created pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Container created 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Container started 11m Normal Started pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Container started 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Add eth0 [10.134.0.207/23] from ovn-kubernetes 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Add eth0 [10.133.1.27/23] from ovn-kubernetes 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Container created 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Warning Unhealthy pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Readiness probe failed: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused 11m Warning ProbeError pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Readiness probe error: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused... 11m Normal Pulled pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Normal Created pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Container created 11m Normal Started pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Container started 11m Warning ProbeError pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Startup probe error: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused... 11m Warning Unhealthy pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Startup probe failed: Get "http://10.133.1.26:15021/healthz/ready": dial tcp 10.133.1.26:15021: connect: connection refused 10m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-096081-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-096081-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Stopping container payload-pre-processing 10m Normal Killing pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Stopping container payload-processing 10m Normal Killing pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Stopping container istio-proxy 10m Warning ProbeError pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/e2e-worker-single-main-096081-openshift-default-5bf85f6d49s7btz Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Readiness probe failed: Get "http://10.133.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-8xkxn Readiness probe error: Get "http://10.133.1.13:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Readiness probe failed: Get "http://10.134.0.205:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 10m Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-fp2b7 Readiness probe error: Get "http://10.134.0.205:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-096081-74b89c648hcqp7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-096081-7d87c2dmwr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m2s Normal AddedLabel gateway/e2e-worker-single-main-3813a1 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-3813a1 8m1s Normal ScalingReplicaSet deployment/e2e-worker-single-main-3813a1-openshift-default Scaled up replica set e2e-worker-single-main-3813a1-openshift-default-6477bb64c7 from 0 to 1 8m1s Normal AddedInterface pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Add eth0 [10.133.1.33/23] from ovn-kubernetes 8m1s Normal SuccessfulCreate replicaset/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7 Created pod: e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 7m57s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cb6 Created pod: payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 7m57s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cb6 Error creating: pods "payload-processing-e2e-worker-single-main-3813a1-6f6bd99cb6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-3813a1: serviceaccount "payload-processing-e2e-worker-single-main-3813a1" not found 7m57s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-3813a1-74bd74578c Created pod: payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp 7m57s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-3813a1 Scaled up replica set payload-pre-processing-e2e-worker-single-main-3813a1-74bd74578c from 0 to 1 7m57s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-3813a1 Scaled up replica set payload-processing-e2e-worker-single-main-3813a1-6f6bd99cb6 from 0 to 1 7m57s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-3813a1-74bd74578c Error creating: pods "payload-pre-processing-e2e-worker-single-main-3813a1-74bd74578c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-3813a1: serviceaccount "payload-processing-e2e-worker-single-main-3813a1" not found 7m56s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m56s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Add eth0 [10.133.1.34/23] from ovn-kubernetes 7m56s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Add eth0 [10.134.0.209/23] from ovn-kubernetes 7m56s Normal Pulled pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 7m56s Normal Created pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Container created 7m56s Normal Started pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Container started 7m56s Normal Created pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Container created 7m56s Normal Started pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Container started 7m46s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-3813a1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m46s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-3813a1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m44s Normal Started pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Container started 7m44s Warning ProbeError pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Readiness probe error: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused... 7m44s Normal Pulled pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m44s Normal Created pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Container created 7m44s Warning Unhealthy pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Readiness probe failed: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused 7m39s Warning Unhealthy pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Startup probe failed: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused 7m39s Warning ProbeError pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Startup probe error: Get "http://10.133.1.33:15021/healthz/ready": dial tcp 10.133.1.33:15021: connect: connection refused... 7m1s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-3813a1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m1s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-3813a1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 6m50s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Stopping container payload-pre-processing 6m50s Normal Killing pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Stopping container payload-processing 6m42s Normal Killing pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Stopping container istio-proxy 6m38s Warning ProbeError pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Readiness probe error: HTTP probe failed with statuscode: 503... 6m38s Warning Unhealthy pod/e2e-worker-single-main-3813a1-openshift-default-6477bb64c7mz2d8 Readiness probe failed: HTTP probe failed with statuscode: 503 6m35s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m34s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-3813a1-74bd7tpzgp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m24s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-3813a1-6f6bd99cbhd6z8 Readiness probe failed: timeout: failed to connect service "10.134.0.209:50052" within 1s: context deadline exceeded 6m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Readiness probe failed: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m17s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-m9bdz Readiness probe error: Get "http://10.132.0.53:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 6m17s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Readiness probe failed: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 6m17s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-hcdq8 Readiness probe error: Get "http://10.133.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m38s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Readiness probe failed: Get "http://10.133.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m38s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-cjw9g Readiness probe error: Get "http://10.133.1.9:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m38s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Readiness probe error: Get "http://10.133.1.10:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m38s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-gwqw2 Readiness probe failed: Get "http://10.133.1.10:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m38s Warning Unhealthy pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Readiness probe failed: Get "http://10.132.1.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m38s Warning ProbeError pod/maas-default-gateway-openshift-default-5bc884dd87-tmn2k Readiness probe error: Get "http://10.132.1.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m12s Normal AddedLabel gateway/e2e-ait-eaf797bf Added label istio.io/rev=openshift-gateway to gateway e2e-ait-eaf797bf 5m12s Normal ScalingReplicaSet deployment/e2e-ait-eaf797bf-openshift-default Scaled up replica set e2e-ait-eaf797bf-openshift-default-75d787674c from 0 to 1 5m12s Normal SuccessfulCreate replicaset/e2e-ait-eaf797bf-openshift-default-75d787674c Created pod: e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn 5m11s Normal AddedInterface pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Add eth0 [10.134.0.211/23] from ovn-kubernetes 5m8s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-eaf797bf-588c684f59 Created pod: payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv 5m8s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-eaf797bf-588c684f59 Error creating: pods "payload-pre-processing-e2e-ait-eaf797bf-588c684f59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-eaf797bf: serviceaccount "payload-processing-e2e-ait-eaf797bf" not found 5m8s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-eaf797bf Scaled up replica set payload-pre-processing-e2e-ait-eaf797bf-588c684f59 from 0 to 1 5m8s Warning FailedCreate replicaset/payload-processing-e2e-ait-eaf797bf-747856577b Error creating: pods "payload-processing-e2e-ait-eaf797bf-747856577b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-eaf797bf: serviceaccount "payload-processing-e2e-ait-eaf797bf" not found 5m8s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-eaf797bf-747856577b Created pod: payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm 5m8s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-eaf797bf Scaled up replica set payload-processing-e2e-ait-eaf797bf-747856577b from 0 to 1 5m8s Normal AddedInterface pod/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm Add eth0 [10.133.1.42/23] from ovn-kubernetes 5m8s Normal Pulled pod/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m7s Normal Created pod/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv Container created 5m7s Normal AddedInterface pod/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv Add eth0 [10.133.1.43/23] from ovn-kubernetes 5m7s Normal Pulled pod/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m7s Normal Started pod/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv Container started 5m7s Normal Started pod/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm Container started 5m7s Normal Created pod/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm Container created 4m56s Normal Started pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Container started 4m56s Normal Created pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Container created 4m56s Normal Pulled pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m50s Warning Unhealthy pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Startup probe failed: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused 4m50s Warning ProbeError pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Startup probe error: Get "http://10.134.0.211:15021/healthz/ready": dial tcp 10.134.0.211:15021: connect: connection refused... 4m44s Normal Killing pod/payload-pre-processing-e2e-ait-eaf797bf-588c684f59-pn9lv Stopping container payload-pre-processing 4m44s Normal Killing pod/payload-processing-e2e-ait-eaf797bf-747856577b-4h9sm Stopping container payload-processing 4m42s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-eaf797bf-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m42s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-eaf797bf-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m37s Normal Killing pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Stopping container istio-proxy 4m36s Normal AddedLabel gateway/e2e-authcfg-http-1fef08 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-1fef08 4m36s Normal ScalingReplicaSet deployment/e2e-authcfg-http-1fef08-openshift-default Scaled up replica set e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45 from 0 to 1 4m36s Normal SuccessfulCreate replicaset/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45 Created pod: e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n 4m35s Normal AddedInterface pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Add eth0 [10.134.0.212/23] from ovn-kubernetes 4m34s Warning Unhealthy pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Readiness probe failed: HTTP probe failed with statuscode: 503 4m34s Warning ProbeError pod/e2e-ait-eaf797bf-openshift-default-75d787674c-rnwcn Readiness probe error: HTTP probe failed with statuscode: 503... 4m32s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-1fef08 Scaled up replica set payload-processing-e2e-authcfg-http-1fef08-7cb8479996 from 0 to 1 4m31s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f Error creating: pods "payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-1fef08: serviceaccount "payload-processing-e2e-authcfg-http-1fef08" not found 4m31s Normal Pulled pod/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m31s Normal Created pod/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf Container created 4m31s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 Add eth0 [10.133.1.46/23] from ovn-kubernetes 4m31s Normal Created pod/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 Container created 4m31s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-1fef08-7cb8479996 Error creating: pods "payload-processing-e2e-authcfg-http-1fef08-7cb8479996-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-1fef08: serviceaccount "payload-processing-e2e-authcfg-http-1fef08" not found 4m31s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m31s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f Created pod: payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf 4m31s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-1fef08 Scaled up replica set payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f from 0 to 1 4m31s Normal Started pod/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 Container started 4m31s Normal Started pod/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf Container started 4m31s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-1fef08-7cb8479996 Created pod: payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 4m31s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf Add eth0 [10.133.1.45/23] from ovn-kubernetes 4m19s Normal Pulled pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m19s Normal Created pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Container created 4m19s Normal Started pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Container started 4m19s Warning ProbeError pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Readiness probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 4m19s Warning Unhealthy pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Readiness probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 4m13s Warning ProbeError pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Startup probe error: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused... 4m13s Warning Unhealthy pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Startup probe failed: Get "http://10.134.0.212:15021/healthz/ready": dial tcp 10.134.0.212:15021: connect: connection refused 4m6s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-1fef08-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m6s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-1fef08-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 3m51s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-1fef08-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 3m51s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-1fef08-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 2m48s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-1fef08-686d85b69f-hh7bf Stopping container payload-pre-processing 2m48s Normal Killing pod/payload-processing-e2e-authcfg-http-1fef08-7cb8479996-fpf99 Stopping container payload-processing 2m35s Normal Killing pod/e2e-authcfg-http-1fef08-openshift-default-86b4fb9d45-7jv5n Stopping container istio-proxy