=== events in openshift-ingress (2026-10-31T14:22:20Z) === LAST SEEN TYPE REASON OBJECT MESSAGE 28m Normal Scheduled pod/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d Successfully assigned openshift-ingress/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d to ip-10-0-137-77.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c to ip-10-0-137-77.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Successfully assigned openshift-ingress/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w to ip-10-0-128-41.us-east-2.compute.internal 29m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 to ip-10-0-128-41.us-east-2.compute.internal 21m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 to ip-10-0-136-168.us-east-2.compute.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj to ip-10-0-136-168.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d to ip-10-0-136-168.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt to ip-10-0-137-77.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj to ip-10-0-136-168.us-east-2.compute.internal 5m22s Normal Scheduled pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Successfully assigned openshift-ingress/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 Successfully assigned openshift-ingress/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 to ip-10-0-136-168.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Successfully assigned openshift-ingress/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 to ip-10-0-128-41.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 4m40s Normal Scheduled pod/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj Successfully assigned openshift-ingress/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Successfully assigned openshift-ingress/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Successfully assigned openshift-ingress/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd to ip-10-0-128-41.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s to ip-10-0-136-168.us-east-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-5c26q to ip-10-0-137-77.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs Successfully assigned openshift-ingress/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs to ip-10-0-128-41.us-east-2.compute.internal 79m Normal Scheduled pod/router-default-848dcf9575-7trg4 Successfully assigned openshift-ingress/router-default-848dcf9575-7trg4 to ip-10-0-137-77.us-east-2.compute.internal 79m Warning FailedScheduling pod/router-default-848dcf9575-7trg4 no nodes available to schedule pods 51m Normal Scheduled pod/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh Successfully assigned openshift-ingress/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh to ip-10-0-137-77.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh to ip-10-0-136-168.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 to ip-10-0-128-41.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 to ip-10-0-128-41.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 to ip-10-0-128-41.us-east-2.compute.internal 19m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv to ip-10-0-136-168.us-east-2.compute.internal 20m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh to ip-10-0-136-168.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Successfully assigned openshift-ingress/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj to ip-10-0-136-168.us-east-2.compute.internal 25m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z to ip-10-0-128-41.us-east-2.compute.internal 24m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Successfully assigned openshift-ingress/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld to ip-10-0-136-168.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw Successfully assigned openshift-ingress/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw to ip-10-0-136-168.us-east-2.compute.internal 28m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj to ip-10-0-128-41.us-east-2.compute.internal 26m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn to ip-10-0-136-168.us-east-2.compute.internal 23m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw to ip-10-0-137-77.us-east-2.compute.internal 22m Normal Scheduled pod/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc Successfully assigned openshift-ingress/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc to ip-10-0-136-168.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj Successfully assigned openshift-ingress/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj to ip-10-0-128-41.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t to ip-10-0-128-41.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs to ip-10-0-137-77.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Successfully assigned openshift-ingress/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg to ip-10-0-128-41.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw to ip-10-0-128-41.us-east-2.compute.internal 38m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq to ip-10-0-128-41.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x to ip-10-0-136-168.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd to ip-10-0-136-168.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Successfully assigned openshift-ingress/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst to ip-10-0-136-168.us-east-2.compute.internal 41m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 to ip-10-0-128-41.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Successfully assigned openshift-ingress/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj to ip-10-0-137-77.us-east-2.compute.internal 56m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 to ip-10-0-137-77.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl to ip-10-0-128-41.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj to ip-10-0-136-168.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Successfully assigned openshift-ingress/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf to ip-10-0-136-168.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 to ip-10-0-136-168.us-east-2.compute.internal 84m Warning FailedScheduling pod/router-default-848dcf9575-7trg4 no nodes available to schedule pods 47m Normal Scheduled pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 to ip-10-0-136-168.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Successfully assigned openshift-ingress/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj to ip-10-0-128-41.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Successfully assigned openshift-ingress/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 to ip-10-0-136-168.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb to ip-10-0-128-41.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Successfully assigned openshift-ingress/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr to ip-10-0-128-41.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn to ip-10-0-137-77.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 to ip-10-0-137-77.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Successfully assigned openshift-ingress/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj to ip-10-0-136-168.us-east-2.compute.internal 49m Normal Scheduled pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv Successfully assigned openshift-ingress/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv to ip-10-0-136-168.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Successfully assigned openshift-ingress/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q to ip-10-0-136-168.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 38m Warning FailedScheduling pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 4m40s Normal Scheduled pod/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 to ip-10-0-136-168.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Successfully assigned openshift-ingress/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c to ip-10-0-128-41.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 to ip-10-0-136-168.us-east-2.compute.internal 12m Normal Scheduled pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Successfully assigned openshift-ingress/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 to ip-10-0-136-168.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Successfully assigned openshift-ingress/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps to ip-10-0-128-41.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d to ip-10-0-128-41.us-east-2.compute.internal 8m18s Normal Scheduled pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Successfully assigned openshift-ingress/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq to ip-10-0-136-168.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t to ip-10-0-128-41.us-east-2.compute.internal 33m Normal Scheduled pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Successfully assigned openshift-ingress/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv to ip-10-0-137-77.us-east-2.compute.internal 52m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Successfully assigned openshift-ingress/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv to ip-10-0-137-77.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd to ip-10-0-128-41.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Successfully assigned openshift-ingress/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 to ip-10-0-137-77.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Successfully assigned openshift-ingress/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 to ip-10-0-136-168.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 to ip-10-0-137-77.us-east-2.compute.internal 25m Normal Scheduled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Successfully assigned openshift-ingress/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl to ip-10-0-136-168.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Successfully assigned openshift-ingress/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Successfully assigned openshift-ingress/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f to ip-10-0-137-77.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Successfully assigned openshift-ingress/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l to ip-10-0-137-77.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk to ip-10-0-137-77.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp to ip-10-0-137-77.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Successfully assigned openshift-ingress/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z to ip-10-0-136-168.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w to ip-10-0-128-41.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Successfully assigned openshift-ingress/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm Successfully assigned openshift-ingress/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq to ip-10-0-128-41.us-east-2.compute.internal 29m Normal Scheduled pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Successfully assigned openshift-ingress/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j to ip-10-0-137-77.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Successfully assigned openshift-ingress/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n to ip-10-0-136-168.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Successfully assigned openshift-ingress/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh to ip-10-0-137-77.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Successfully assigned openshift-ingress/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp to ip-10-0-136-168.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 to ip-10-0-136-168.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j Successfully assigned openshift-ingress/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j to ip-10-0-136-168.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Successfully assigned openshift-ingress/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf to ip-10-0-128-41.us-east-2.compute.internal 41m Warning FailedScheduling pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 54m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs to ip-10-0-137-77.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Successfully assigned openshift-ingress/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b to ip-10-0-128-41.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Successfully assigned openshift-ingress/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns to ip-10-0-136-168.us-east-2.compute.internal 37m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj to ip-10-0-136-168.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Successfully assigned openshift-ingress/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns to ip-10-0-128-41.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Successfully assigned openshift-ingress/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q to ip-10-0-128-41.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz to ip-10-0-128-41.us-east-2.compute.internal 44m Normal Scheduled pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Successfully assigned openshift-ingress/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h to ip-10-0-128-41.us-east-2.compute.internal 5m18s Normal Scheduled pod/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Successfully assigned openshift-ingress/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq Successfully assigned openshift-ingress/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq to ip-10-0-137-77.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Successfully assigned openshift-ingress/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f to ip-10-0-136-168.us-east-2.compute.internal 48m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Successfully assigned openshift-ingress/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs to ip-10-0-136-168.us-east-2.compute.internal 58m Normal Scheduled pod/payload-pre-processing-7b98bb678c-km7m9 Successfully assigned openshift-ingress/payload-pre-processing-7b98bb678c-km7m9 to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk to ip-10-0-128-41.us-east-2.compute.internal 46m Normal Scheduled pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v to ip-10-0-136-168.us-east-2.compute.internal 47m Normal Scheduled pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Successfully assigned openshift-ingress/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst to ip-10-0-137-77.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Successfully assigned openshift-ingress/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j to ip-10-0-137-77.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d to ip-10-0-128-41.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Successfully assigned openshift-ingress/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Successfully assigned openshift-ingress/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn to ip-10-0-136-168.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Successfully assigned openshift-ingress/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 to ip-10-0-137-77.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 35m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt to ip-10-0-128-41.us-east-2.compute.internal 36m Normal Scheduled pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Successfully assigned openshift-ingress/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb to ip-10-0-136-168.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns to ip-10-0-136-168.us-east-2.compute.internal 55m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Successfully assigned openshift-ingress/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 to ip-10-0-137-77.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh to ip-10-0-128-41.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g to ip-10-0-137-77.us-east-2.compute.internal 4m45s Normal Scheduled pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Successfully assigned openshift-ingress/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 to ip-10-0-136-168.us-east-2.compute.internal 50m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 to ip-10-0-128-41.us-east-2.compute.internal 73m Normal Scheduled pod/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 Successfully assigned openshift-ingress/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 to ip-10-0-137-77.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Successfully assigned openshift-ingress/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw to ip-10-0-137-77.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Successfully assigned openshift-ingress/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz to ip-10-0-128-41.us-east-2.compute.internal 47m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Successfully assigned openshift-ingress/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Successfully assigned openshift-ingress/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr to ip-10-0-137-77.us-east-2.compute.internal 73m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 to ip-10-0-137-77.us-east-2.compute.internal 34m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Successfully assigned openshift-ingress/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 to ip-10-0-136-168.us-east-2.compute.internal 8m14s Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Successfully assigned openshift-ingress/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc to ip-10-0-128-41.us-east-2.compute.internal 12m Normal Scheduled pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Successfully assigned openshift-ingress/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 to ip-10-0-128-41.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv to ip-10-0-128-41.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Successfully assigned openshift-ingress/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm to ip-10-0-137-77.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 to ip-10-0-137-77.us-east-2.compute.internal 46m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Successfully assigned openshift-ingress/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 to ip-10-0-136-168.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg to ip-10-0-137-77.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt to ip-10-0-137-77.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Successfully assigned openshift-ingress/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Successfully assigned openshift-ingress/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Successfully assigned openshift-ingress/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb to ip-10-0-136-168.us-east-2.compute.internal 41m Normal Scheduled pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Successfully assigned openshift-ingress/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 to ip-10-0-137-77.us-east-2.compute.internal 41m Warning FailedScheduling pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Successfully assigned openshift-ingress/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Successfully assigned openshift-ingress/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp to ip-10-0-136-168.us-east-2.compute.internal 57m Normal Scheduled pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Successfully assigned openshift-ingress/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Successfully assigned openshift-ingress/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn to ip-10-0-137-77.us-east-2.compute.internal 58m Normal Scheduled pod/payload-processing-55fbbbf9dc-w9wjk Successfully assigned openshift-ingress/payload-processing-55fbbbf9dc-w9wjk to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Successfully assigned openshift-ingress/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 to ip-10-0-136-168.us-east-2.compute.internal 20m Normal Scheduled pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Successfully assigned openshift-ingress/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs Successfully assigned openshift-ingress/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs to ip-10-0-136-168.us-east-2.compute.internal 39m Normal Scheduled pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Successfully assigned openshift-ingress/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b to ip-10-0-137-77.us-east-2.compute.internal 39m Warning FailedScheduling pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 57m Normal Scheduled pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Successfully assigned openshift-ingress/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq to ip-10-0-128-41.us-east-2.compute.internal 12m Normal Scheduled pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv to ip-10-0-136-168.us-east-2.compute.internal 8m14s Normal Scheduled pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Successfully assigned openshift-ingress/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s to ip-10-0-136-168.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq to ip-10-0-137-77.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 to ip-10-0-128-41.us-east-2.compute.internal 21m Normal Scheduled pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Successfully assigned openshift-ingress/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 to ip-10-0-137-77.us-east-2.compute.internal 54m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Successfully assigned openshift-ingress/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh to ip-10-0-136-168.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Successfully assigned openshift-ingress/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk to ip-10-0-128-41.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Successfully assigned openshift-ingress/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq to ip-10-0-137-77.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh to ip-10-0-136-168.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 to ip-10-0-137-77.us-east-2.compute.internal 48m Normal Scheduled pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Successfully assigned openshift-ingress/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn to ip-10-0-137-77.us-east-2.compute.internal 53m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk to ip-10-0-128-41.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Successfully assigned openshift-ingress/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r to ip-10-0-128-41.us-east-2.compute.internal 50m Normal Scheduled pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Successfully assigned openshift-ingress/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z to ip-10-0-128-41.us-east-2.compute.internal 57m Normal Scheduled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Successfully assigned openshift-ingress/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj to ip-10-0-136-168.us-east-2.compute.internal 39m Warning FailedScheduling pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 47m Normal Scheduled pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 to ip-10-0-136-168.us-east-2.compute.internal 5m18s Normal Scheduled pod/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs Successfully assigned openshift-ingress/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v Successfully assigned openshift-ingress/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v to ip-10-0-136-168.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-ait-dff70365-577bb65985-lv664 Successfully assigned openshift-ingress/payload-processing-e2e-ait-dff70365-577bb65985-lv664 to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Successfully assigned openshift-ingress/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f to ip-10-0-137-77.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 to ip-10-0-136-168.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv to ip-10-0-128-41.us-east-2.compute.internal 38m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Successfully assigned openshift-ingress/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 to ip-10-0-136-168.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 to ip-10-0-128-41.us-east-2.compute.internal 34m Normal Scheduled pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Successfully assigned openshift-ingress/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr to ip-10-0-136-168.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Successfully assigned openshift-ingress/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr to ip-10-0-128-41.us-east-2.compute.internal 26m Normal Scheduled pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Successfully assigned openshift-ingress/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv to ip-10-0-137-77.us-east-2.compute.internal 73m Normal Scheduled pod/istiod-openshift-gateway-7876bfcb74-ltcvs Successfully assigned openshift-ingress/istiod-openshift-gateway-7876bfcb74-ltcvs to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww to ip-10-0-136-168.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-5z724 to ip-10-0-137-77.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn to ip-10-0-128-41.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr to ip-10-0-128-41.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs to ip-10-0-136-168.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk to ip-10-0-128-41.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz Successfully assigned openshift-ingress/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz to ip-10-0-128-41.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 to ip-10-0-128-41.us-east-2.compute.internal 58m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-srpzw to ip-10-0-128-41.us-east-2.compute.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 to ip-10-0-128-41.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-66bvk to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 to ip-10-0-137-77.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-59fjz to ip-10-0-137-77.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f to ip-10-0-128-41.us-east-2.compute.internal 53m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-bb782 to ip-10-0-128-41.us-east-2.compute.internal 31m Normal Scheduled pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Successfully assigned openshift-ingress/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 to ip-10-0-136-168.us-east-2.compute.internal 25m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-gk567 to ip-10-0-128-41.us-east-2.compute.internal 71m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-thzmx Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-thzmx to ip-10-0-136-168.us-east-2.compute.internal 37m Normal Scheduled pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Successfully assigned openshift-ingress/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx to ip-10-0-137-77.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 to ip-10-0-137-77.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 to ip-10-0-128-41.us-east-2.compute.internal 24m Normal Scheduled pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Successfully assigned openshift-ingress/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj to ip-10-0-136-168.us-east-2.compute.internal 50m Normal Scheduled pod/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d to ip-10-0-128-41.us-east-2.compute.internal 35m Normal Scheduled pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m to ip-10-0-128-41.us-east-2.compute.internal 37m Normal Scheduled pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Successfully assigned openshift-ingress/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 to ip-10-0-136-168.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-n56bm to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p to ip-10-0-137-77.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r to ip-10-0-128-41.us-east-2.compute.internal 50m Normal Scheduled pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Successfully assigned openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-wb6gv to ip-10-0-136-168.us-east-2.compute.internal 34m Normal Scheduled pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Successfully assigned openshift-ingress/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Successfully assigned openshift-ingress/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 to ip-10-0-137-77.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk Successfully assigned openshift-ingress/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 to ip-10-0-137-77.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb to ip-10-0-136-168.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 Successfully assigned openshift-ingress/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj to ip-10-0-137-77.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r to ip-10-0-128-41.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc to ip-10-0-137-77.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd to ip-10-0-128-41.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m to ip-10-0-137-77.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 to ip-10-0-137-77.us-east-2.compute.internal 49m Normal Scheduled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Successfully assigned openshift-ingress/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz to ip-10-0-136-168.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx to ip-10-0-128-41.us-east-2.compute.internal 51m Normal Scheduled pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Successfully assigned openshift-ingress/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r to ip-10-0-128-41.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k Successfully assigned openshift-ingress/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k to ip-10-0-136-168.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 Successfully assigned openshift-ingress/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Successfully assigned openshift-ingress/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh to ip-10-0-136-168.us-east-2.compute.internal 49m Normal Scheduled pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Successfully assigned openshift-ingress/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt to ip-10-0-128-41.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 to ip-10-0-128-41.us-east-2.compute.internal 16m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w to ip-10-0-137-77.us-east-2.compute.internal 48m Normal Scheduled pod/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b Successfully assigned openshift-ingress/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b to ip-10-0-136-168.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 39m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl to ip-10-0-128-41.us-east-2.compute.internal 23m Normal Scheduled pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Successfully assigned openshift-ingress/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt to ip-10-0-136-168.us-east-2.compute.internal 72m Normal Scheduled pod/kube-auth-proxy-7485898565-4fxqz Successfully assigned openshift-ingress/kube-auth-proxy-7485898565-4fxqz to ip-10-0-136-168.us-east-2.compute.internal 47m Normal Scheduled pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h Successfully assigned openshift-ingress/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h to ip-10-0-128-41.us-east-2.compute.internal 45m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k to ip-10-0-136-168.us-east-2.compute.internal 28m Normal Scheduled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Successfully assigned openshift-ingress/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn to ip-10-0-128-41.us-east-2.compute.internal 59m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz to ip-10-0-128-41.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s to ip-10-0-136-168.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 41m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 to ip-10-0-128-41.us-east-2.compute.internal 46m Normal Scheduled pod/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb Successfully assigned openshift-ingress/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb to ip-10-0-128-41.us-east-2.compute.internal 72m Normal Scheduled pod/kube-auth-proxy-7485898565-kp66f Successfully assigned openshift-ingress/kube-auth-proxy-7485898565-kp66f to ip-10-0-128-41.us-east-2.compute.internal 17m Normal Scheduled pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Successfully assigned openshift-ingress/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Successfully assigned openshift-ingress/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk to ip-10-0-137-77.us-east-2.compute.internal 51m Normal Scheduled pod/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p Successfully assigned openshift-ingress/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p to ip-10-0-136-168.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 to ip-10-0-136-168.us-east-2.compute.internal 22m Normal Scheduled pod/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc to ip-10-0-128-41.us-east-2.compute.internal 42m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 40m Normal Scheduled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Successfully assigned openshift-ingress/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff to ip-10-0-128-41.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw to ip-10-0-128-41.us-east-2.compute.internal 72m Normal Scheduled pod/data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf Successfully assigned openshift-ingress/data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf to ip-10-0-137-77.us-east-2.compute.internal 38m Warning FailedScheduling pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 0/3 nodes are available: 3 Insufficient cpu. no new claims to deallocate, preemption: 0/3 nodes are available: 3 No preemption victims found for incoming pod. 27m Normal Scheduled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Successfully assigned openshift-ingress/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm to ip-10-0-137-77.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz to ip-10-0-136-168.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws to ip-10-0-137-77.us-east-2.compute.internal 25m Normal Scheduled pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch to ip-10-0-128-41.us-east-2.compute.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 to ip-10-0-136-168.us-east-2.compute.internal 19m Normal Scheduled pod/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz to ip-10-0-128-41.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz to ip-10-0-128-41.us-east-2.compute.internal 26m Normal Scheduled pod/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f to ip-10-0-128-41.us-east-2.compute.internal 23m Normal Scheduled pod/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 to ip-10-0-136-168.us-east-2.compute.internal 24m Normal Scheduled pod/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 to ip-10-0-136-168.us-east-2.compute.internal 20m Normal Scheduled pod/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt to ip-10-0-136-168.us-east-2.compute.internal 21m Normal Scheduled pod/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc to ip-10-0-136-168.us-east-2.compute.internal 27m Normal Scheduled pod/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 to ip-10-0-128-41.us-east-2.compute.internal 29m Normal Scheduled pod/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 to ip-10-0-128-41.us-east-2.compute.internal 28m Normal Scheduled pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Successfully assigned openshift-ingress/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk to ip-10-0-128-41.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg to ip-10-0-128-41.us-east-2.compute.internal 38m Normal Scheduled pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs to ip-10-0-136-168.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb to ip-10-0-128-41.us-east-2.compute.internal 41m Normal Scheduled pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 to ip-10-0-128-41.us-east-2.compute.internal 43m Normal Scheduled pod/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r Successfully assigned openshift-ingress/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r to ip-10-0-128-41.us-east-2.compute.internal 44m Normal Scheduled pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Successfully assigned openshift-ingress/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj to ip-10-0-136-168.us-east-2.compute.internal 44m Normal Scheduled pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n to ip-10-0-136-168.us-east-2.compute.internal 45m Normal Scheduled pod/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 Successfully assigned openshift-ingress/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 to ip-10-0-136-168.us-east-2.compute.internal 84m Normal ScalingReplicaSet deployment/router-default Scaled up replica set router-default-848dcf9575 from 0 to 1 84m Normal SuccessfulCreate replicaset/router-default-848dcf9575 Created pod: router-default-848dcf9575-7trg4 84m Warning UnAvailableLoadBalancer service/router-default There are no available nodes for LoadBalancer 84m Normal EnsuringLoadBalancer service/router-default Ensuring load balancer 84m Normal EnsuredLoadBalancer service/router-default Ensured load balancer 79m Normal UpdatedLoadBalancer service/router-default Updated load balancer with new hosts 79m Warning FailedMount pod/router-default-848dcf9575-7trg4 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 79m Warning FailedMount pod/router-default-848dcf9575-7trg4 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 78m Warning FailedMount pod/router-default-848dcf9575-7trg4 MountVolume.SetUp failed for volume "service-ca-bundle" : configmap references non-existent config key: service-ca.crt 78m Warning FailedMount pod/router-default-848dcf9575-7trg4 MountVolume.SetUp failed for volume "metrics-certs" : secret "router-metrics-certs-default" not found 78m Normal Started pod/router-default-848dcf9575-7trg4 Container started 78m Normal Created pod/router-default-848dcf9575-7trg4 Container created 78m Normal Pulled pod/router-default-848dcf9575-7trg4 Container image "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3a679d80eb36ca7db888d4fbd1fdf16b0997a7c7a107936396b7a57d8b199f4e" already present on machine and can be accessed by the pod 78m Normal AddedInterface pod/router-default-848dcf9575-7trg4 Add eth0 [10.133.0.12/23] from ovn-kubernetes 73m Normal AddedLabel gateway/maas-default-gateway Added label istio.io/rev=openshift-gateway to gateway maas-default-gateway 73m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7876bfcb74 from 0 to 1 73m Normal AddedInterface pod/istiod-openshift-gateway-7876bfcb74-ltcvs Add eth0 [10.134.0.15/23] from ovn-kubernetes 73m Normal Pulling pod/istiod-openshift-gateway-7876bfcb74-ltcvs Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 73m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7876bfcb74 Created pod: istiod-openshift-gateway-7876bfcb74-ltcvs 73m Normal Created pod/istiod-openshift-gateway-7876bfcb74-ltcvs Container created 73m Normal Started pod/istiod-openshift-gateway-7876bfcb74-ltcvs Container started 73m Normal Pulled pod/istiod-openshift-gateway-7876bfcb74-ltcvs Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.386s (3.386s including waiting). Image size: 185476477 bytes. 73m Normal EnsuringLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensuring load balancer 73m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 73m Normal ScalingReplicaSet deployment/openshift-ai-inference-openshift-ai-inference Scaled up replica set openshift-ai-inference-openshift-ai-inference-54b87d5f7f from 0 to 1 73m Normal SuccessfulCreate replicaset/openshift-ai-inference-openshift-ai-inference-54b87d5f7f Created pod: openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 73m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5d84bc6b95 from 0 to 1 73m Normal AddedInterface pod/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 Add eth0 [10.133.0.33/23] from ovn-kubernetes 73m Normal Pulling pod/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 73m Normal Pulling pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 73m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Add eth0 [10.133.0.32/23] from ovn-kubernetes 73m Normal EnsuredLoadBalancer service/openshift-ai-inference-openshift-ai-inference Ensured load balancer 73m Normal Pulled pod/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.02s (13.02s including waiting). Image size: 183555333 bytes. 73m Normal Created pod/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 Container created 73m Normal Started pod/openshift-ai-inference-openshift-ai-inference-54b87d5f7f-qt2c6 Container started 73m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 13.079s (13.079s including waiting). Image size: 183555333 bytes. 73m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 73m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 72m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 72m Warning FailedGetResourceMetric horizontalpodautoscaler/openshift-ai-inference-openshift-ai-inference failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 72m Normal ScalingReplicaSet deployment/data-science-gateway-data-science-gateway-class Scaled up replica set data-science-gateway-data-science-gateway-class-6bbdd9f58f from 0 to 1 72m Normal SuccessfulCreate replicaset/kube-auth-proxy-7485898565 Created pod: kube-auth-proxy-7485898565-4fxqz 72m Normal SuccessfulCreate replicaset/data-science-gateway-data-science-gateway-class-6bbdd9f58f Created pod: data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf 72m Normal SuccessfulCreate replicaset/kube-auth-proxy-7485898565 Created pod: kube-auth-proxy-7485898565-kp66f 72m Normal ScalingReplicaSet deployment/kube-auth-proxy Scaled up replica set kube-auth-proxy-7485898565 from 0 to 2 72m Normal AddedInterface pod/kube-auth-proxy-7485898565-4fxqz Add eth0 [10.134.0.18/23] from ovn-kubernetes 72m Normal AddedInterface pod/data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf Add eth0 [10.133.0.43/23] from ovn-kubernetes 72m Normal Pulling pod/kube-auth-proxy-7485898565-4fxqz Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 72m Normal Pulled pod/data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 72m Normal AddedInterface pod/kube-auth-proxy-7485898565-kp66f Add eth0 [10.132.0.31/23] from ovn-kubernetes 72m Normal Started pod/data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf Container started 72m Normal Pulling pod/kube-auth-proxy-7485898565-kp66f Pulling image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" 72m Normal Created pod/data-science-gateway-data-science-gateway-class-6bbdd9f58fn2phf Container created 71m Normal Created pod/kube-auth-proxy-7485898565-kp66f Container created 71m Normal Started pod/kube-auth-proxy-7485898565-4fxqz Container started 71m Normal Created pod/kube-auth-proxy-7485898565-4fxqz Container created 71m Normal Pulled pod/kube-auth-proxy-7485898565-4fxqz Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.353s (2.353s including waiting). Image size: 188904677 bytes. 71m Normal Started pod/kube-auth-proxy-7485898565-kp66f Container started 71m Normal Pulled pod/kube-auth-proxy-7485898565-kp66f Successfully pulled image "quay.io/opendatahub/odh-kube-auth-proxy@sha256:d90930380151490d5516eeb95840eecc743e9005fc02733dec6fe79e1a00e035" in 2.438s (2.438s including waiting). Image size: 188904677 bytes. 71m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/kube-auth-proxy failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/kube-auth-proxy invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 71m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/data-science-gateway-data-science-gateway-class invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Warning FailedGetResourceMetric horizontalpodautoscaler/data-science-gateway-data-science-gateway-class failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 71m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 0 to 1 71m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-thzmx 71m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-thzmx Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 71m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-thzmx Container created 71m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-thzmx Container started 71m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-thzmx Add eth0 [10.134.0.22/23] from ovn-kubernetes 71m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7876bfcb74 Deleted pod: istiod-openshift-gateway-7876bfcb74-ltcvs 71m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7876bfcb74 from 1 to 0 71m Normal Killing pod/istiod-openshift-gateway-7876bfcb74-ltcvs Stopping container discovery 70m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/istiod-openshift-gateway invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 70m Warning FailedGetResourceMetric horizontalpodautoscaler/istiod-openshift-gateway failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Container created 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Container started 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Startup probe failed: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Startup probe error: Get "http://10.133.0.32:15021/healthz/ready": dial tcp 10.133.0.32:15021: connect: connection refused... 59m Warning FailedGetResourceMetric horizontalpodautoscaler/maas-default-gateway-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/maas-default-gateway-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 59m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 59m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5d84bc6b95 from 1 to 4 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-66bvk 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-vcml7 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-dlwlz 59m Normal Pulling pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Add eth0 [10.134.0.30/23] from ovn-kubernetes 59m Normal Pulling pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 59m Normal Pulling pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Pulling image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Add eth0 [10.132.0.34/23] from ovn-kubernetes 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Add eth0 [10.134.0.29/23] from ovn-kubernetes 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Container started 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.123s (3.123s including waiting). Image size: 183555333 bytes. 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Container created 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Container started 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Container created 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 3.25s (3.25s including waiting). Image size: 183555333 bytes. 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Container started 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Container created 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" in 5.861s (5.861s including waiting). Image size: 183555333 bytes. 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Startup probe failed: Get "http://10.134.0.29:15021/healthz/ready": dial tcp 10.134.0.29:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Startup probe error: Get "http://10.134.0.29:15021/healthz/ready": dial tcp 10.134.0.29:15021: connect: connection refused... 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Startup probe error: Get "http://10.134.0.30:15021/healthz/ready": dial tcp 10.134.0.30:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Startup probe failed: Get "http://10.134.0.30:15021/healthz/ready": dial tcp 10.134.0.30:15021: connect: connection refused 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Startup probe failed: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Startup probe error: Get "http://10.132.0.34:15021/healthz/ready": dial tcp 10.132.0.34:15021: connect: connection refused... 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-vl7kq 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-hl6bc 59m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 59m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-n56bm 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Container created 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Container created 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Container started 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Add eth0 [10.134.0.31/23] from ovn-kubernetes 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Add eth0 [10.133.0.56/23] from ovn-kubernetes 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Container started 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Add eth0 [10.132.0.36/23] from ovn-kubernetes 59m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Add eth0 [10.132.0.35/23] from ovn-kubernetes 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Container created 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Container started 59m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Container created 59m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Container started 59m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Startup probe error: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Startup probe failed: Get "http://10.134.0.31:15021/healthz/ready": dial tcp 10.134.0.31:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Startup probe error: Get "http://10.132.0.36:15021/healthz/ready": dial tcp 10.132.0.36:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Startup probe failed: Get "http://10.132.0.35:15021/healthz/ready": dial tcp 10.132.0.35:15021: connect: connection refused 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Startup probe failed: Get "http://10.132.0.36:15021/healthz/ready": dial tcp 10.132.0.36:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Startup probe error: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused... 59m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Startup probe failed: Get "http://10.133.0.56:15021/healthz/ready": dial tcp 10.133.0.56:15021: connect: connection refused 59m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Startup probe error: Get "http://10.132.0.35:15021/healthz/ready": dial tcp 10.132.0.35:15021: connect: connection refused... 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-5c26q 58m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: 58m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-srpzw 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Add eth0 [10.132.0.37/23] from ovn-kubernetes 58m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Container created 58m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Container started 58m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Container started 58m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Container created 58m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 58m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Add eth0 [10.133.0.57/23] from ovn-kubernetes 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Startup probe failed: Get "http://10.132.0.37:15021/healthz/ready": dial tcp 10.132.0.37:15021: connect: connection refused 58m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": dial tcp 10.133.0.57:15021: connect: connection refused... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Startup probe error: Get "http://10.132.0.37:15021/healthz/ready": dial tcp 10.132.0.37:15021: connect: connection refused... 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Startup probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Startup probe failed: Get "http://10.132.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 58m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Startup probe error: Get "http://10.132.0.37:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Startup probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 58m Warning FailedCreate replicaset/payload-processing-55fbbbf9dc Error creating: pods "payload-processing-55fbbbf9dc-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 58m Warning FailedCreate replicaset/payload-pre-processing-7b98bb678c Error creating: pods "payload-pre-processing-7b98bb678c-" is forbidden: error looking up service account openshift-ingress/payload-processing: serviceaccount "payload-processing" not found 58m Normal ScalingReplicaSet deployment/payload-pre-processing Scaled up replica set payload-pre-processing-7b98bb678c from 0 to 1 58m Normal ScalingReplicaSet deployment/payload-processing Scaled up replica set payload-processing-55fbbbf9dc from 0 to 1 58m Normal SuccessfulCreate replicaset/payload-pre-processing-7b98bb678c Created pod: payload-pre-processing-7b98bb678c-km7m9 58m Normal SuccessfulCreate replicaset/payload-processing-55fbbbf9dc Created pod: payload-processing-55fbbbf9dc-w9wjk 58m Normal Pulling pod/payload-processing-55fbbbf9dc-w9wjk Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal AddedInterface pod/payload-processing-55fbbbf9dc-w9wjk Add eth0 [10.134.0.39/23] from ovn-kubernetes 58m Normal AddedInterface pod/payload-pre-processing-7b98bb678c-km7m9 Add eth0 [10.134.0.38/23] from ovn-kubernetes 58m Normal Pulling pod/payload-pre-processing-7b98bb678c-km7m9 Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 58m Normal Created pod/payload-pre-processing-7b98bb678c-km7m9 Container created 58m Normal Started pod/payload-processing-55fbbbf9dc-w9wjk Container started 58m Normal Started pod/payload-pre-processing-7b98bb678c-km7m9 Container started 58m Normal Pulled pod/payload-pre-processing-7b98bb678c-km7m9 Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.334s (2.334s including waiting). Image size: 125353963 bytes. 58m Normal Pulled pod/payload-processing-55fbbbf9dc-w9wjk Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.253s (2.253s including waiting). Image size: 125353963 bytes. 58m Normal Created pod/payload-processing-55fbbbf9dc-w9wjk Container created 57m Normal AddedLabel gateway/e2e-worker-w2-5286ea Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w2-5286ea 57m Normal AddedLabel gateway/e2e-worker-w3-c82140 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w3-c82140 57m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns 57m Normal AddedLabel gateway/e2e-worker-w6-aabeef Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w6-aabeef 57m Normal AddedLabel gateway/e2e-worker-w1-d85d82 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w1-d85d82 57m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled up replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 0 to 1 57m Normal AddedLabel gateway/e2e-worker-w4-8b2f0a Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w4-8b2f0a 57m Normal AddedLabel gateway/e2e-worker-w5-6784d6 Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w5-6784d6 57m Normal AddedLabel gateway/e2e-worker-w0-7b3ddd Added label istio.io/rev=openshift-gateway to gateway e2e-worker-w0-7b3ddd 57m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled up replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 0 to 1 57m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled up replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 0 to 1 57m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Add eth0 [10.134.0.41/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj 57m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled up replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj 57m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Add eth0 [10.132.0.42/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 57m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled up replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 0 to 1 57m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled up replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq 57m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj 57m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled up replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 0 to 1 57m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld 57m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Add eth0 [10.132.0.41/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Add eth0 [10.134.0.43/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Add eth0 [10.134.0.42/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Add eth0 [10.134.0.44/23] from ovn-kubernetes 57m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Add eth0 [10.132.0.43/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w3-c82140 Scaled up replica set payload-processing-e2e-worker-w3-c82140-575b9b7477 from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w5-6784d6-6dfd744559 Error creating: pods "payload-processing-e2e-worker-w5-6784d6-6dfd744559-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-6784d6: serviceaccount "payload-processing-e2e-worker-w5-6784d6" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w5-6784d6-6dfd744559 Created pod: payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w5-6784d6 Scaled up replica set payload-processing-e2e-worker-w5-6784d6-6dfd744559 from 0 to 1 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w5-6784d6 Scaled up replica set payload-pre-processing-e2e-worker-w5-6784d6-d5845499 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w5-6784d6-d5845499 Created pod: payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w5-6784d6-d5845499 Error creating: pods "payload-pre-processing-e2e-worker-w5-6784d6-d5845499-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w5-6784d6: serviceaccount "payload-processing-e2e-worker-w5-6784d6" not found 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w3-c82140-575b9b7477 Created pod: payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Add eth0 [10.134.0.46/23] from ovn-kubernetes 57m Normal Pulling pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w1-d85d82-655545f7d7 Created pod: payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w1-d85d82 Scaled up replica set payload-processing-e2e-worker-w1-d85d82-655545f7d7 from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff Error creating: pods "payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-d85d82: serviceaccount "payload-processing-e2e-worker-w1-d85d82" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff Created pod: payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w1-d85d82 Scaled up replica set payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff from 0 to 1 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w3-c82140-575b9b7477 Error creating: pods "payload-processing-e2e-worker-w3-c82140-575b9b7477-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-c82140: serviceaccount "payload-processing-e2e-worker-w3-c82140" not found 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w1-d85d82-655545f7d7 Error creating: pods "payload-processing-e2e-worker-w1-d85d82-655545f7d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w1-d85d82: serviceaccount "payload-processing-e2e-worker-w1-d85d82" not found 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Add eth0 [10.134.0.45/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Add eth0 [10.132.0.44/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Pulling pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Container created 57m Normal Created pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Container started 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w3-c82140 Scaled up replica set payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55 Created pod: payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55 Error creating: pods "payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w3-c82140: serviceaccount "payload-processing-e2e-worker-w3-c82140" not found 57m Normal Started pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Container started 57m Normal AddedInterface pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Add eth0 [10.133.0.59/23] from ovn-kubernetes 57m Normal Started pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Container started 57m Normal Created pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Container created 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Add eth0 [10.132.0.46/23] from ovn-kubernetes 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5 Created pod: payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq 57m Normal Pulling pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w4-8b2f0a Scaled up replica set payload-processing-e2e-worker-w4-8b2f0a-544f99cf49 from 0 to 1 57m Normal Pulling pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49 Created pod: payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49 Error creating: pods "payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-8b2f0a: serviceaccount "payload-processing-e2e-worker-w4-8b2f0a" not found 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9 Error creating: pods "payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w4-8b2f0a: serviceaccount "payload-processing-e2e-worker-w4-8b2f0a" not found 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5 Error creating: pods "payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-7b3ddd: serviceaccount "payload-processing-e2e-worker-w0-7b3ddd" not found 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w0-7b3ddd Scaled up replica set payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9 Created pod: payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w0-7b3ddd Scaled up replica set payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5 from 0 to 1 57m Normal AddedInterface pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Add eth0 [10.132.0.45/23] from ovn-kubernetes 57m Normal AddedInterface pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Add eth0 [10.132.0.47/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5 Error creating: pods "payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w0-7b3ddd: serviceaccount "payload-processing-e2e-worker-w0-7b3ddd" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5 Created pod: payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv 57m Normal Pulling pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Add eth0 [10.134.0.47/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w4-8b2f0a Scaled up replica set payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9 from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67 Created pod: payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w6-aabeef Scaled up replica set payload-processing-e2e-worker-w6-aabeef-7fbf76c96c from 0 to 1 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67 Error creating: pods "payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-5286ea: serviceaccount "payload-processing-e2e-worker-w2-5286ea" not found 57m Normal Pulled pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c Error creating: pods "payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-aabeef: serviceaccount "payload-processing-e2e-worker-w6-aabeef" not found 57m Normal AddedInterface pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Add eth0 [10.134.0.50/23] from ovn-kubernetes 57m Warning FailedCreate replicaset/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c Error creating: pods "payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w2-5286ea: serviceaccount "payload-processing-e2e-worker-w2-5286ea" not found 57m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c Created pod: payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w2-5286ea Scaled up replica set payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67 from 0 to 1 57m Normal Pulling pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c Created pod: payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc 57m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-w6-aabeef Scaled up replica set payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c from 0 to 1 57m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c Created pod: payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp 57m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c Error creating: pods "payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-w6-aabeef: serviceaccount "payload-processing-e2e-worker-w6-aabeef" not found 57m Normal AddedInterface pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Add eth0 [10.134.0.48/23] from ovn-kubernetes 57m Normal Pulled pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Created pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Container created 57m Normal Started pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Container started 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Add eth0 [10.132.0.48/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Add eth0 [10.134.0.49/23] from ovn-kubernetes 57m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-w2-5286ea Scaled up replica set payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c from 0 to 1 57m Normal Pulled pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 3.424s (3.424s including waiting). Image size: 125353963 bytes. 57m Normal Created pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Container created 57m Normal AddedInterface pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Add eth0 [10.132.0.49/23] from ovn-kubernetes 57m Normal Created pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Container created 57m Normal Started pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Container started 57m Normal Pulling pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Pulling image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" 57m Normal Pulled pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 391ms (391ms including waiting). Image size: 125353963 bytes. 57m Normal Created pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Container created 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.074s (2.075s including waiting). Image size: 125353963 bytes. 57m Normal Started pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Container started 57m Normal Created pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Container started 57m Normal AddedInterface pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Add eth0 [10.134.0.51/23] from ovn-kubernetes 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 57m Normal Pulled pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.632s (2.632s including waiting). Image size: 125353963 bytes. 57m Normal Pulled pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.365s (2.365s including waiting). Image size: 125353963 bytes. 57m Normal Created pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Container created 57m Normal Created pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Container started 57m Normal Started pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Container started 57m Normal Created pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Container created 57m Normal Started pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Container started 57m Normal Started pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Container started 57m Normal Created pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Container created 57m Normal Created pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Container created 57m Normal Created pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Container created 57m Normal Started pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Container started 57m Normal Started pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Container started 57m Normal Started pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Container started 57m Normal Pulled pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 1.787s (1.787s including waiting). Image size: 125353963 bytes. 57m Normal Pulled pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Successfully pulled image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" in 2.133s (2.133s including waiting). Image size: 125353963 bytes. 57m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Readiness probe failed: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 57m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Readiness probe error: Get "http://10.134.0.42:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 57m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Container created 57m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Container started 57m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Container created 57m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Container created 57m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Container started 57m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Container started 57m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Container created 57m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Container started 57m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Container created 57m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Container started 57m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Container created 57m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Container started 57m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Container started 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 57m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Container created 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 57m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 57m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Startup probe failed: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused 57m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Startup probe failed: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Startup probe error: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Startup probe failed: Get "http://10.134.0.43:15021/healthz/ready": dial tcp 10.134.0.43:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Startup probe error: Get "http://10.134.0.44:15021/healthz/ready": dial tcp 10.134.0.44:15021: connect: connection refused... 57m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Startup probe error: Get "http://10.134.0.42:15021/healthz/ready": dial tcp 10.134.0.42:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Startup probe failed: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Startup probe error: Get "http://10.134.0.41:15021/healthz/ready": dial tcp 10.134.0.41:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Startup probe failed: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Startup probe error: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Startup probe failed: Get "http://10.132.0.43:15021/healthz/ready": dial tcp 10.132.0.43:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Startup probe error: Get "http://10.132.0.42:15021/healthz/ready": dial tcp 10.132.0.42:15021: connect: connection refused... 57m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Startup probe failed: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused 57m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Startup probe error: Get "http://10.132.0.41:15021/healthz/ready": dial tcp 10.132.0.41:15021: connect: connection refused... 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 56m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 56m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 56m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled up replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 1 to 2 56m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 56m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Add eth0 [10.133.0.68/23] from ovn-kubernetes 56m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Container started 56m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Container created 56m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Startup probe failed: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused 56m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Startup probe error: Get "http://10.133.0.68:15021/healthz/ready": dial tcp 10.133.0.68:15021: connect: connection refused... 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj 55m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Add eth0 [10.132.0.57/23] from ovn-kubernetes 55m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Container created 55m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Container started 55m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h 55m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled up replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 1 to 2 55m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled up replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 1 to 2 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Container started 55m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Container created 55m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Add eth0 [10.134.0.58/23] from ovn-kubernetes 55m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled up replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 1 to 2 55m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Add eth0 [10.134.0.59/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Container created 55m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Container started 55m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Startup probe failed: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Startup probe error: Get "http://10.134.0.59:15021/healthz/ready": dial tcp 10.134.0.59:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Startup probe error: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused... 55m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Startup probe error: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused... 55m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Startup probe failed: Get "http://10.132.0.57:15021/healthz/ready": dial tcp 10.132.0.57:15021: connect: connection refused 55m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Startup probe failed: Get "http://10.134.0.58:15021/healthz/ready": dial tcp 10.134.0.58:15021: connect: connection refused 55m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled up replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 2 to 4 55m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 55m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled up replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 1 to 2 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb 55m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 55m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Container started 55m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Container started 55m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Add eth0 [10.134.0.66/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Container created 55m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Add eth0 [10.133.0.70/23] from ovn-kubernetes 55m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Container created 55m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 55m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Add eth0 [10.132.0.66/23] from ovn-kubernetes 55m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Container started 55m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Container created 55m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Startup probe failed: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused 55m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Startup probe error: Get "http://10.134.0.66:15021/healthz/ready": dial tcp 10.134.0.66:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Startup probe error: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Startup probe error: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Startup probe failed: Get "http://10.133.0.70:15021/healthz/ready": dial tcp 10.133.0.70:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Startup probe failed: Get "http://10.132.0.66:15021/healthz/ready": dial tcp 10.132.0.66:15021: connect: connection refused 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled up replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 1 to 2 54m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled up replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 2 to 3 54m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp 54m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc 54m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled up replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 1 to 2 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t 54m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled up replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 2 to 4 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 54m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Add eth0 [10.133.0.73/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Container created 54m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Add eth0 [10.132.0.70/23] from ovn-kubernetes 54m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Container started 54m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Container created 54m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Container started 54m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Container started 54m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Container created 54m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Add eth0 [10.133.0.74/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Add eth0 [10.132.0.72/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Container created 54m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Container started 54m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Container started 54m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Add eth0 [10.132.0.71/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Container created 54m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Startup probe failed: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Startup probe error: Get "http://10.132.0.70:15021/healthz/ready": dial tcp 10.132.0.70:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Startup probe failed: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Startup probe error: Get "http://10.133.0.73:15021/healthz/ready": dial tcp 10.133.0.73:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Startup probe failed: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Startup probe error: Get "http://10.133.0.74:15021/healthz/ready": dial tcp 10.133.0.74:15021: connect: connection refused... 54m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Startup probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Startup probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Startup probe failed: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Startup probe error: Get "http://10.132.0.72:15021/healthz/ready": dial tcp 10.132.0.72:15021: connect: connection refused... 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled up replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 2 to 3 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Add eth0 [10.133.0.76/23] from ovn-kubernetes 54m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Container created 54m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Container started 54m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Startup probe failed: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Startup probe error: Get "http://10.133.0.76:15021/healthz/ready": dial tcp 10.133.0.76:15021: connect: connection refused... 54m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled up replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 3 to 4 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 54m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Add eth0 [10.134.0.71/23] from ovn-kubernetes 54m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled up replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 2 to 3 54m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns 54m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr 54m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled up replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 2 to 3 54m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled up replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 4 to 5 54m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Container started 54m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Add eth0 [10.133.0.77/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Add eth0 [10.134.0.72/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Container created 54m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Container started 54m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Add eth0 [10.132.0.76/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Container created 54m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Container started 54m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Container created 54m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Container started 54m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Container created 54m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused 54m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": dial tcp 10.132.0.76:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": dial tcp 10.134.0.71:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Startup probe failed: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Startup probe error: Get "http://10.134.0.72:15021/healthz/ready": dial tcp 10.134.0.72:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused 54m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": dial tcp 10.133.0.77:15021: connect: connection refused... 54m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Startup probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Startup probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Startup probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Startup probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Readiness probe error: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Readiness probe error: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Startup probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 54m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Startup probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Readiness probe failed: Get "http://10.133.0.74:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Readiness probe failed: Get "http://10.132.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 54m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx 54m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r 54m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled up replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 3 to 5 54m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Container started 54m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Container created 54m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Add eth0 [10.132.0.78/23] from ovn-kubernetes 54m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Add eth0 [10.134.0.74/23] from ovn-kubernetes 54m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 54m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Container created 54m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Container started 53m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Startup probe failed: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Startup probe failed: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Startup probe error: Get "http://10.132.0.78:15021/healthz/ready": dial tcp 10.132.0.78:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Startup probe error: Get "http://10.134.0.74:15021/healthz/ready": dial tcp 10.134.0.74:15021: connect: connection refused... 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled up replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 2 to 4 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 6; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled up replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 4 to 6 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled up replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 4 to 6 53m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Add eth0 [10.133.0.80/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Container created 53m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Add eth0 [10.132.0.79/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Add eth0 [10.134.0.76/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Container started 53m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Add eth0 [10.133.0.81/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Add eth0 [10.133.0.82/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Add eth0 [10.134.0.77/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Container created 53m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Container started 53m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Container started 53m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Container created 53m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Container created 53m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Container created 53m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Container started 53m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Container started 53m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Container started 53m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Container created 53m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Startup probe failed: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Startup probe error: Get "http://10.132.0.79:15021/healthz/ready": dial tcp 10.132.0.79:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Startup probe failed: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Startup probe error: Get "http://10.134.0.76:15021/healthz/ready": dial tcp 10.134.0.76:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Startup probe failed: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Startup probe error: Get "http://10.134.0.77:15021/healthz/ready": dial tcp 10.134.0.77:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Startup probe failed: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Startup probe error: Get "http://10.133.0.82:15021/healthz/ready": dial tcp 10.133.0.82:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Startup probe error: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Startup probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Startup probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Startup probe failed: Get "http://10.133.0.80:15021/healthz/ready": dial tcp 10.133.0.80:15021: connect: connection refused 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled up replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 3 to 4 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt 53m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled up replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 5 to 8 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c 53m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled up replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 3 to 5 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq 53m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j 53m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 7; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled up replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 5 to 7 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Container started 53m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Container created 53m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Container created 53m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Container started 53m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Add eth0 [10.132.0.84/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Add eth0 [10.133.0.83/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Container created 53m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Container started 53m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Container started 53m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Container started 53m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Add eth0 [10.134.0.83/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Container started 53m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Container started 53m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Add eth0 [10.134.0.81/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Container created 53m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Add eth0 [10.133.0.84/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Container created 53m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Add eth0 [10.134.0.80/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Add eth0 [10.134.0.82/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Container created 53m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Container created 53m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Container created 53m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Add eth0 [10.132.0.82/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Container started 53m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 1 to 2 53m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-bb782 53m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Startup probe error: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused... 53m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Add eth0 [10.132.0.85/23] from ovn-kubernetes 53m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Startup probe failed: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Startup probe error: Get "http://10.133.0.83:15021/healthz/ready": dial tcp 10.133.0.83:15021: connect: connection refused... 53m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 53m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Startup probe failed: Get "http://10.132.0.82:15021/healthz/ready": dial tcp 10.132.0.82:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Startup probe error: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused... 53m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Startup probe error: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Startup probe failed: Get "http://10.132.0.84:15021/healthz/ready": dial tcp 10.132.0.84:15021: connect: connection refused 53m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Startup probe failed: Get "http://10.133.0.84:15021/healthz/ready": dial tcp 10.133.0.84:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Startup probe error: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused... 53m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Startup probe failed: Get "http://10.134.0.80:15021/healthz/ready": dial tcp 10.134.0.80:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Startup probe error: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused... 53m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 3.63s (3.63s including waiting). Image size: 185476477 bytes. 53m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Startup probe failed: Get "http://10.134.0.81:15021/healthz/ready": dial tcp 10.134.0.81:15021: connect: connection refused 53m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Container started 53m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Container created 53m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Startup probe failed: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Startup probe error: Get "http://10.134.0.82:15021/healthz/ready": dial tcp 10.134.0.82:15021: connect: connection refused... 53m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Readiness probe failed: HTTP probe failed with statuscode: 503 53m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Readiness probe error: HTTP probe failed with statuscode: 503... 53m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Startup probe failed: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused 53m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Startup probe error: Get "http://10.134.0.83:15021/healthz/ready": dial tcp 10.134.0.83:15021: connect: connection refused... 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 9; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh 53m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled up replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 6 to 10 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c (combined from similar events): Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk 53m Normal SuccessfulCreate replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Created pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn 53m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled up replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 4 to 8 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled up replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 7 to 10 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd 53m Normal SuccessfulCreate replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 (combined from similar events): Created pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj 53m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled up replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 6 to 9 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 53m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq 53m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled up replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 4 to 5 53m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 5; reason: cpu resource utilization (percentage of request) above target 53m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Container created 53m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Container created 53m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Container started 53m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Container started 53m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Container created 53m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Container started 53m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Container created 53m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Add eth0 [10.134.0.89/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Add eth0 [10.134.0.88/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Container started 53m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Container created 53m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Add eth0 [10.133.0.89/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Container created 53m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Container started 53m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Container started 53m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Add eth0 [10.133.0.86/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Add eth0 [10.134.0.90/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Add eth0 [10.133.0.91/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Container created 53m Normal Pulled pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Add eth0 [10.132.0.89/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Add eth0 [10.132.0.92/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Container created 53m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Add eth0 [10.133.0.87/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Container created 53m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Container started 53m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm 53m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Add eth0 [10.133.0.88/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Container started 53m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Container created 53m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Container started 53m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Add eth0 [10.133.0.90/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Container created 53m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Add eth0 [10.132.0.91/23] from ovn-kubernetes 53m Normal Started pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Container started 53m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Container started 53m Normal Started pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Container started 53m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Container created 53m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Add eth0 [10.132.0.90/23] from ovn-kubernetes 53m Normal Created pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Container created 53m Normal AddedInterface pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Add eth0 [10.134.0.91/23] from ovn-kubernetes 53m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Add eth0 [10.134.0.92/23] from ovn-kubernetes 53m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Container created 53m Normal Pulled pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 53m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Container started 53m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Container started 53m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Container created 53m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Container started 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Startup probe failed: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Startup probe error: Get "http://10.132.0.89:15021/healthz/ready": dial tcp 10.132.0.89:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Startup probe error: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Startup probe failed: Get "http://10.132.0.90:15021/healthz/ready": dial tcp 10.132.0.90:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Startup probe failed: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Startup probe error: Get "http://10.134.0.88:15021/healthz/ready": dial tcp 10.134.0.88:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Startup probe error: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Startup probe failed: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Startup probe error: Get "http://10.132.0.92:15021/healthz/ready": dial tcp 10.132.0.92:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Startup probe failed: Get "http://10.132.0.91:15021/healthz/ready": dial tcp 10.132.0.91:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Startup probe failed: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Startup probe error: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Startup probe error: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Startup probe failed: Get "http://10.134.0.89:15021/healthz/ready": dial tcp 10.134.0.89:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Startup probe failed: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Startup probe failed: Get "http://10.134.0.92:15021/healthz/ready": dial tcp 10.134.0.92:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Startup probe error: Get "http://10.134.0.90:15021/healthz/ready": dial tcp 10.134.0.90:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Startup probe error: Get "http://10.134.0.91:15021/healthz/ready": dial tcp 10.134.0.91:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Startup probe failed: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Startup probe failed: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Startup probe failed: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Startup probe failed: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": dial tcp 10.133.0.86:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Startup probe failed: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Startup probe failed: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Startup probe error: Get "http://10.133.0.87:15021/healthz/ready": dial tcp 10.133.0.87:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 (combined from similar events): Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Startup probe error: Get "http://10.133.0.86:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Startup probe error: Get "http://10.133.0.88:15021/healthz/ready": dial tcp 10.133.0.88:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Startup probe error: Get "http://10.133.0.89:15021/healthz/ready": dial tcp 10.133.0.89:15021: connect: connection refused... 52m Normal SuccessfulCreate replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Created pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Startup probe error: Get "http://10.133.0.90:15021/healthz/ready": dial tcp 10.133.0.90:15021: connect: connection refused... 52m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled up replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 8 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j 52m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled up replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 9 to 10 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled up replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 8 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc (combined from similar events): Created pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Startup probe error: Get "http://10.133.0.91:15021/healthz/ready": dial tcp 10.133.0.91:15021: connect: connection refused... 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 10; reason: All metrics below target 52m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Container created 52m Normal Pulled pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Container started 52m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Add eth0 [10.134.0.93/23] from ovn-kubernetes 52m Normal Created pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Container created 52m Normal SuccessfulCreate replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c (combined from similar events): Created pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld 52m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Add eth0 [10.132.0.93/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Container started 52m Normal Created pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Container created 52m Normal Pulled pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Add eth0 [10.134.0.95/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Container created 52m Normal Started pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Container started 52m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Container started 52m Normal AddedInterface pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Add eth0 [10.134.0.94/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Container started 52m Normal AddedInterface pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Add eth0 [10.133.0.92/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Container created 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Readiness probe error: Get "http://10.132.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Readiness probe failed: Get "http://10.132.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Readiness probe error: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Readiness probe failed: Get "http://10.133.0.73:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Readiness probe failed: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Readiness probe failed: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Readiness probe failed: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Readiness probe failed: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Readiness probe error: Get "http://10.133.0.70:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Readiness probe error: Get "http://10.132.0.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Readiness probe error: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Readiness probe failed: Get "http://10.132.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Readiness probe error: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Readiness probe failed: Get "http://10.134.0.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Readiness probe error: Get "http://10.132.0.84:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/payload-processing-55fbbbf9dc-w9wjk Readiness probe failed: timeout: failed to connect service "10.134.0.39:50052" within 1s: context deadline exceeded 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Readiness probe error: Get "http://10.132.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Readiness probe error: Get "http://10.134.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Readiness probe error: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Readiness probe failed: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Readiness probe error: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Readiness probe error: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Readiness probe failed: Get "http://10.134.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Readiness probe failed: Get "http://10.132.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Readiness probe failed: Get "http://10.133.0.77:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Readiness probe failed: Get "http://10.134.0.72:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Readiness probe error: Get "http://10.134.0.66:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Readiness probe error: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Readiness probe failed: Get "http://10.133.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Readiness probe error: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Readiness probe failed: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Readiness probe error: Get "http://10.134.0.80:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Readiness probe failed: Get "http://10.134.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Startup probe failed: Get "http://10.132.0.93:15021/healthz/ready": dial tcp 10.132.0.93:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Startup probe failed: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Startup probe failed: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Startup probe failed: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Readiness probe failed: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Startup probe error: Get "http://10.134.0.94:15021/healthz/ready": dial tcp 10.134.0.94:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Startup probe error: Get "http://10.134.0.93:15021/healthz/ready": dial tcp 10.134.0.93:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Startup probe error: Get "http://10.134.0.95:15021/healthz/ready": dial tcp 10.134.0.95:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Readiness probe failed: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Startup probe error: Get "http://10.132.0.93:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Startup probe failed: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Startup probe error: Get "http://10.133.0.92:15021/healthz/ready": dial tcp 10.133.0.92:15021: connect: connection refused... 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled up replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 5 to 10 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb 52m Normal SuccessfulCreate replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf (combined from similar events): Created pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f (combined from similar events): Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd 52m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Container created 52m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Container created 52m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Container started 52m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Container created 52m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Container started 52m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Container created 52m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Add eth0 [10.133.0.94/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Container created 52m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Container started 52m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Add eth0 [10.134.0.103/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Container started 52m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Container started 52m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Container created 52m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Add eth0 [10.133.0.93/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Container created 52m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Container started 52m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Add eth0 [10.132.0.99/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Container created 52m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Container started 52m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Container started 52m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled up replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 5 to 10 52m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Add eth0 [10.134.0.100/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal Created pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Container created 52m Normal Started pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Container started 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 52m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 10; reason: cpu resource utilization (percentage of request) above target 52m Normal Pulled pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Add eth0 [10.132.0.100/23] from ovn-kubernetes 52m Normal SuccessfulCreate replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Created pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw 52m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Add eth0 [10.132.0.98/23] from ovn-kubernetes 52m Normal Pulled pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 52m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Add eth0 [10.134.0.102/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Add eth0 [10.133.0.95/23] from ovn-kubernetes 52m Normal AddedInterface pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Add eth0 [10.134.0.101/23] from ovn-kubernetes 52m Normal Started pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Container started 52m Normal Created pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Container created 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Readiness probe failed: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Readiness probe error: Get "http://10.132.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Readiness probe failed: timeout: failed to connect service "10.134.0.51:50052" within 1s: context deadline exceeded 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Readiness probe error: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Readiness probe error: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Readiness probe failed: Get "http://10.133.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Readiness probe failed: Get "http://10.132.0.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Readiness probe error: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Readiness probe failed: Get "http://10.132.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Readiness probe error: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Readiness probe failed: Get "http://10.134.0.71:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Readiness probe failed: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Readiness probe failed: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Readiness probe error: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Readiness probe error: Get "http://10.134.0.76:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Readiness probe error: Get "http://10.132.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Readiness probe failed: Get "http://10.132.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Startup probe error: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Startup probe failed: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Startup probe failed: Get "http://10.132.0.99:15021/healthz/ready": dial tcp 10.132.0.99:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Startup probe failed: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Startup probe failed: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Startup probe error: Get "http://10.132.0.98:15021/healthz/ready": dial tcp 10.132.0.98:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Startup probe failed: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Readiness probe error: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Startup probe failed: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Startup probe error: Get "http://10.134.0.100:15021/healthz/ready": dial tcp 10.134.0.100:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Startup probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Startup probe error: Get "http://10.134.0.103:15021/healthz/ready": dial tcp 10.134.0.103:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Readiness probe failed: Get "http://10.134.0.82:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Readiness probe error: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Startup probe failed: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Startup probe error: Get "http://10.133.0.93:15021/healthz/ready": dial tcp 10.133.0.93:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Readiness probe failed: Get "http://10.134.0.89:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Readiness probe failed: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Readiness probe error: Get "http://10.134.0.90:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Startup probe failed: Get "http://10.134.0.103:15021/healthz/ready": dial tcp 10.134.0.103:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Startup probe error: Get "http://10.132.0.100:15021/healthz/ready": dial tcp 10.132.0.100:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Readiness probe error: Get "http://10.134.0.88:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Startup probe failed: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused 52m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Startup probe failed: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused 52m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Readiness probe error: Get "http://10.134.0.91:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Readiness probe error: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Startup probe error: Get "http://10.133.0.95:15021/healthz/ready": dial tcp 10.133.0.95:15021: connect: connection refused... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Startup probe error: Get "http://10.134.0.103:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 52m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Startup probe error: Get "http://10.133.0.94:15021/healthz/ready": dial tcp 10.133.0.94:15021: connect: connection refused... 52m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Readiness probe failed: Get "http://10.134.0.83:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 52m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Startup probe error: Get "http://10.134.0.101:15021/healthz/ready": dial tcp 10.134.0.101:15021: connect: connection refused... 51m Normal ScalingReplicaSet deployment/e2e-derive-c7b990ba-openshift-default Scaled up replica set e2e-derive-c7b990ba-openshift-default-69444bf76f from 0 to 1 51m Normal AddedLabel gateway/e2e-derive-c7b990ba Added label istio.io/rev=openshift-gateway to gateway e2e-derive-c7b990ba 51m Normal SuccessfulCreate replicaset/e2e-derive-c7b990ba-openshift-default-69444bf76f Created pod: e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 51m Normal AddedInterface pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Add eth0 [10.132.0.112/23] from ovn-kubernetes 51m Warning FailedCreate replicaset/payload-processing-e2e-derive-c7b990ba-6d4457fc6f Error creating: pods "payload-processing-e2e-derive-c7b990ba-6d4457fc6f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c7b990ba: serviceaccount "payload-processing-e2e-derive-c7b990ba" not found 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59 Error creating: pods "payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-derive-c7b990ba: serviceaccount "payload-processing-e2e-derive-c7b990ba" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-derive-c7b990ba Scaled up replica set payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59 from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-derive-c7b990ba Scaled up replica set payload-processing-e2e-derive-c7b990ba-6d4457fc6f from 0 to 1 51m Normal ScalingReplicaSet deployment/e2e-ait-8439b78a-openshift-default Scaled up replica set e2e-ait-8439b78a-openshift-default-5789ddb9c from 0 to 1 51m Normal AddedLabel gateway/e2e-ait-8439b78a Added label istio.io/rev=openshift-gateway to gateway e2e-ait-8439b78a 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-derive-c7b990ba-6d4457fc6f Created pod: payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59 Created pod: payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv 51m Normal SuccessfulCreate replicaset/e2e-ait-8439b78a-openshift-default-5789ddb9c Created pod: e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d 51m Normal AddedInterface pod/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv Add eth0 [10.134.0.110/23] from ovn-kubernetes 51m Normal Started pod/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 Container started 51m Normal AddedInterface pod/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 Add eth0 [10.134.0.111/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv Container created 51m Normal Started pod/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv Container started 51m Normal Created pod/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 Container created 51m Normal AddedInterface pod/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d Add eth0 [10.133.0.98/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedLabel gateway/e2e-migrate-fc1bbe2b-gw Added label istio.io/rev=openshift-gateway to gateway e2e-migrate-fc1bbe2b-gw 51m Normal ScalingReplicaSet deployment/e2e-migrate-fc1bbe2b-gw-openshift-default Scaled up replica set e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657 Created pod: e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r 51m Normal AddedInterface pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Add eth0 [10.132.0.113/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-8439b78a Scaled up replica set payload-processing-e2e-ait-8439b78a-7c455dfdc5 from 0 to 1 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-8439b78a Scaled up replica set payload-pre-processing-e2e-ait-8439b78a-7496664b99 from 0 to 1 51m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Readiness probe failed: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Readiness probe error: Get "http://10.133.0.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-8439b78a-7496664b99 Created pod: payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-8439b78a-7496664b99 Error creating: pods "payload-pre-processing-e2e-ait-8439b78a-7496664b99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8439b78a: serviceaccount "payload-processing-e2e-ait-8439b78a" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-8439b78a-7c455dfdc5 Error creating: pods "payload-processing-e2e-ait-8439b78a-7c455dfdc5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-8439b78a: serviceaccount "payload-processing-e2e-ait-8439b78a" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-8439b78a-7c455dfdc5 Created pod: payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs 51m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Readiness probe error: Get "http://10.134.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Readiness probe failed: timeout: health rpc did not complete within 1s 51m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Readiness probe failed: Get "http://10.134.0.92:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c7b990ba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Readiness probe failed: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Readiness probe error: Get "http://10.132.0.99:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Readiness probe failed: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Readiness probe error: Get "http://10.132.0.98:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c7b990ba-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Pulled pod/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq Add eth0 [10.133.0.99/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-migrate-fc1bbe2b Scaled up replica set payload-processing-e2e-migrate-fc1bbe2b-55f58c5867 from 0 to 1 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867 Created pod: payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595 Created pod: payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj 51m Warning FailedCreate replicaset/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867 Error creating: pods "payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-fc1bbe2b: serviceaccount "payload-processing-e2e-migrate-fc1bbe2b" not found 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-migrate-fc1bbe2b Scaled up replica set payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595 from 0 to 1 51m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Readiness probe error: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595 Error creating: pods "payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-migrate-fc1bbe2b: serviceaccount "payload-processing-e2e-migrate-fc1bbe2b" not found 51m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Readiness probe failed: Get "http://10.134.0.41:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 51m Normal Created pod/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq Container created 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8439b78a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8439b78a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal AddedInterface pod/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj Add eth0 [10.132.0.115/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj Container started 51m Normal Started pod/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq Container started 51m Normal Created pod/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj Container created 51m Normal Pulled pod/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Pulled pod/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedLabel gateway/e2e-ait-a66ae960 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a66ae960 51m Normal AddedInterface pod/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs Add eth0 [10.134.0.114/23] from ovn-kubernetes 51m Normal AddedInterface pod/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh Add eth0 [10.133.0.100/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-ait-a66ae960-openshift-default Scaled up replica set e2e-ait-a66ae960-openshift-default-5bdd6f848f from 0 to 1 51m Normal Pulled pod/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/e2e-ait-a66ae960-openshift-default-5bdd6f848f Created pod: e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-fc1bbe2b-gw-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Started pod/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs Container started 51m Normal Created pod/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs Container created 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-fc1bbe2b-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Started pod/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p Container started 51m Normal Created pod/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p Container created 51m Normal AddedInterface pod/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p Add eth0 [10.134.0.115/23] from ovn-kubernetes 51m Normal Pulled pod/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh Container created 51m Normal Started pod/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh Container started 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-a66ae960-b7f949887 Created pod: payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-a66ae960-f4d46755 Created pod: payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-a66ae960 Scaled up replica set payload-pre-processing-e2e-ait-a66ae960-b7f949887 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-a66ae960-b7f949887 Error creating: pods "payload-pre-processing-e2e-ait-a66ae960-b7f949887-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a66ae960: serviceaccount "payload-processing-e2e-ait-a66ae960" not found 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-a66ae960-f4d46755 Error creating: pods "payload-processing-e2e-ait-a66ae960-f4d46755-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a66ae960: serviceaccount "payload-processing-e2e-ait-a66ae960" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-a66ae960 Scaled up replica set payload-processing-e2e-ait-a66ae960-f4d46755 from 0 to 1 51m Normal Started pod/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v Container started 51m Normal Pulled pod/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Started pod/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h Container started 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h Add eth0 [10.132.0.116/23] from ovn-kubernetes 51m Normal Created pod/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h Container created 51m Normal Pulled pod/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Created pod/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v Container created 51m Normal AddedInterface pod/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v Add eth0 [10.134.0.117/23] from ovn-kubernetes 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-derive-c7b990ba-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-derive-c7b990ba-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal SuccessfulCreate replicaset/e2e-mt-a8c1600c-openshift-default-74b4bf4f85 Created pod: e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 51m Normal AddedLabel gateway/e2e-ait-dff70365 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-dff70365 51m Normal AddedLabel gateway/e2e-mt-a8c1600c Added label istio.io/rev=openshift-gateway to gateway e2e-mt-a8c1600c 51m Normal ScalingReplicaSet deployment/e2e-mt-a8c1600c-openshift-default Scaled up replica set e2e-mt-a8c1600c-openshift-default-74b4bf4f85 from 0 to 1 51m Normal SuccessfulCreate replicaset/e2e-ait-dff70365-openshift-default-7b549865c5 Created pod: e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw 51m Normal AddedInterface pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Add eth0 [10.133.0.101/23] from ovn-kubernetes 51m Normal ScalingReplicaSet deployment/e2e-ait-dff70365-openshift-default Scaled up replica set e2e-ait-dff70365-openshift-default-7b549865c5 from 0 to 1 51m Normal Created pod/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw Container created 51m Normal Started pod/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw Container started 51m Normal Pulled pod/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw Add eth0 [10.134.0.118/23] from ovn-kubernetes 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-8439b78a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-8439b78a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-a8c1600c-764447d479 Created pod: payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-a8c1600c Scaled up replica set payload-pre-processing-e2e-mt-a8c1600c-764447d479 from 0 to 1 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-a8c1600c-764447d479 Error creating: pods "payload-pre-processing-e2e-mt-a8c1600c-764447d479-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a8c1600c: serviceaccount "payload-processing-e2e-mt-a8c1600c" not found 51m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-59fjz 51m Warning FailedCreate replicaset/payload-processing-e2e-mt-a8c1600c-7d96b7788c Error creating: pods "payload-processing-e2e-mt-a8c1600c-7d96b7788c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-a8c1600c: serviceaccount "payload-processing-e2e-mt-a8c1600c" not found 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-a8c1600c-7d96b7788c Created pod: payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-a8c1600c Scaled up replica set payload-processing-e2e-mt-a8c1600c-7d96b7788c from 0 to 1 51m Normal AddedInterface pod/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 Add eth0 [10.132.0.119/23] from ovn-kubernetes 51m Warning FailedCreate replicaset/payload-processing-e2e-ait-dff70365-577bb65985 Error creating: pods "payload-processing-e2e-ait-dff70365-577bb65985-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dff70365: serviceaccount "payload-processing-e2e-ait-dff70365" not found 51m Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-dff70365 Scaled up replica set payload-processing-e2e-ait-dff70365-577bb65985 from 0 to 1 51m Normal Created pod/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm Container created 51m Normal Pulled pod/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal Pulled pod/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-dff70365-577bb65985 Created pod: payload-processing-e2e-ait-dff70365-577bb65985-lv664 51m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-dff70365 Scaled up replica set payload-pre-processing-e2e-ait-dff70365-ddf54c767 from 0 to 1 51m Normal Created pod/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 Container created 51m Normal Started pod/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 Container started 51m Normal Started pod/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm Container started 51m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-dff70365-ddf54c767 Created pod: payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b 51m Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-dff70365-ddf54c767 Error creating: pods "payload-pre-processing-e2e-ait-dff70365-ddf54c767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-dff70365: serviceaccount "payload-processing-e2e-ait-dff70365" not found 51m Normal Pulling pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Pulling image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" 51m Normal AddedInterface pod/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm Add eth0 [10.134.0.119/23] from ovn-kubernetes 51m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Add eth0 [10.133.0.102/23] from ovn-kubernetes 51m Normal Pulled pod/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b Add eth0 [10.132.0.120/23] from ovn-kubernetes 51m Normal Started pod/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b Container started 51m Normal Created pod/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b Container created 51m Normal Started pod/payload-processing-e2e-ait-dff70365-577bb65985-lv664 Container started 51m Normal Pulled pod/payload-processing-e2e-ait-dff70365-577bb65985-lv664 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 51m Normal AddedInterface pod/payload-processing-e2e-ait-dff70365-577bb65985-lv664 Add eth0 [10.132.0.121/23] from ovn-kubernetes 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a66ae960-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a66ae960-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Normal Created pod/payload-processing-e2e-ait-dff70365-577bb65985-lv664 Container created 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-migrate-fc1bbe2b-gw-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-migrate-fc1bbe2b-gw-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 51m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Container created 51m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Container started 51m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Successfully pulled image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" in 4.236s (4.236s including waiting). Image size: 185476477 bytes. 51m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Readiness probe failed: HTTP probe failed with statuscode: 503 51m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-59fjz Readiness probe error: HTTP probe failed with statuscode: 503... 51m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dff70365-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dff70365-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 51m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Readiness probe error: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 51m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Readiness probe failed: Get "http://10.134.0.44:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Readiness probe failed: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a66ae960-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Readiness probe error: Get "http://10.134.0.59:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a66ae960-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal Killing pod/payload-processing-e2e-derive-c7b990ba-6d4457fc6f-ww829 Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-derive-c7b990ba-57bfc7df59-vlnxv Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-ait-8439b78a-7c455dfdc5-z24rs Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-ait-8439b78a-7496664b99-2krxq Stopping container payload-pre-processing 50m Normal Killing pod/payload-pre-processing-e2e-migrate-fc1bbe2b-fc58f6595-7wwxj Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-migrate-fc1bbe2b-55f58c5867-7ns2p Stopping container payload-processing 50m Warning ProbeError pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Readiness probe error: Get "http://10.132.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Warning Unhealthy pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Readiness probe failed: Get "http://10.132.0.113:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-dff70365-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-dff70365-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Warning Unhealthy pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Readiness probe failed: Get "http://10.132.0.112:15021/healthz/ready": read tcp 10.132.0.2:46396->10.132.0.112:15021: read: connection reset by peer 50m Warning ProbeError pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Readiness probe error: Get "http://10.132.0.112:15021/healthz/ready": read tcp 10.132.0.2:46396->10.132.0.112:15021: read: connection reset by peer... 50m Normal Pulled pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Container created 50m Normal Pulled pod/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Pulled pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Container created 50m Normal Started pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Container started 50m Normal Started pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Container started 50m Normal Killing pod/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d Stopping container istio-proxy 50m Normal Created pod/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d Container created 50m Normal ScalingReplicaSet deployment/e2e-ait-8439b78a-openshift-default Scaled up replica set e2e-ait-8439b78a-openshift-default-5789ddb9c from 0 to 1 50m Normal Started pod/e2e-ait-8439b78a-openshift-default-5789ddb9c-bjf9d Container started 50m Warning FailedCreate replicaset/e2e-ait-8439b78a-openshift-default-5789ddb9c Error creating: pods "e2e-ait-8439b78a-openshift-default-5789ddb9c-" is forbidden: error looking up service account openshift-ingress/e2e-ait-8439b78a-openshift-default: serviceaccount "e2e-ait-8439b78a-openshift-default" not found 50m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-wb6gv 50m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 4 50m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 4; reason: cpu resource utilization (percentage of request) above target 50m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 50m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Add eth0 [10.134.0.123/23] from ovn-kubernetes 50m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Container started 50m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Container created 50m Warning ProbeError pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Startup probe error: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused... 50m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Readiness probe failed: HTTP probe failed with statuscode: 503 50m Warning ProbeError pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Startup probe error: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused... 50m Normal AddedLabel gateway/e2e-shared-a-w2-4f340c Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w2-4f340c 50m Warning Unhealthy pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Startup probe failed: Get "http://10.132.0.113:15021/healthz/ready": dial tcp 10.132.0.113:15021: connect: connection refused 50m Normal Killing pod/e2e-migrate-fc1bbe2b-gw-openshift-default-6fd96c6657-dqz8r Stopping container istio-proxy 50m Warning Unhealthy pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Startup probe failed: Get "http://10.132.0.112:15021/healthz/ready": dial tcp 10.132.0.112:15021: connect: connection refused 50m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Readiness probe error: HTTP probe failed with statuscode: 503... 50m Normal SuccessfulCreate replicaset/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c Created pod: e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z 50m Normal Killing pod/e2e-derive-c7b990ba-openshift-default-69444bf76f-m4gr4 Stopping container istio-proxy 50m Normal ScalingReplicaSet deployment/e2e-shared-a-w2-4f340c-openshift-default Scaled up replica set e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c from 0 to 1 50m Normal AddedInterface pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Add eth0 [10.132.0.126/23] from ovn-kubernetes 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w2-4f340c Scaled up replica set payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475 from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475 Created pod: payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475 Error creating: pods "payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-4f340c: serviceaccount "payload-processing-e2e-shared-a-w2-4f340c" not found 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w2-4f340c Scaled up replica set payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc from 0 to 1 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc Created pod: payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc Error creating: pods "payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w2-4f340c: serviceaccount "payload-processing-e2e-shared-a-w2-4f340c" not found 50m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Pulled pod/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 50m Normal Started pod/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 Container started 50m Normal Created pod/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d Container created 50m Normal AddedInterface pod/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d Add eth0 [10.132.0.128/23] from ovn-kubernetes 50m Normal Started pod/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d Container started 50m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 Add eth0 [10.132.0.127/23] from ovn-kubernetes 50m Normal Created pod/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 Container created 50m Normal Killing pod/payload-processing-e2e-ait-a66ae960-f4d46755-jxq5v Stopping container payload-processing 50m Normal Killing pod/payload-pre-processing-e2e-ait-a66ae960-b7f949887-nv98h Stopping container payload-pre-processing 50m Normal Killing pod/payload-pre-processing-e2e-ait-dff70365-ddf54c767-g6p6b Stopping container payload-pre-processing 50m Normal Killing pod/payload-processing-e2e-ait-dff70365-577bb65985-lv664 Stopping container payload-processing 50m Normal Killing pod/e2e-ait-a66ae960-openshift-default-5bdd6f848f-7vwwh Stopping container istio-proxy 50m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Readiness probe failed: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 50m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Readiness probe error: Get "http://10.133.0.56:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 50m Normal Killing pod/e2e-ait-dff70365-openshift-default-7b549865c5-f7zxw Stopping container istio-proxy 50m Normal Started pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Container started 50m Normal Pulled pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 50m Normal Created pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Container created 50m Normal SuccessfulCreate replicaset/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5 Created pod: e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw 50m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a8c1600c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal ScalingReplicaSet deployment/e2e-shared-b-w2-b06b08-openshift-default Scaled up replica set e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5 from 0 to 1 50m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a8c1600c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 50m Normal AddedLabel gateway/e2e-shared-b-w2-b06b08 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w2-b06b08 50m Normal AddedInterface pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Add eth0 [10.132.0.130/23] from ovn-kubernetes 50m Warning Unhealthy pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Startup probe failed: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused 50m Warning ProbeError pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Startup probe error: Get "http://10.133.0.101:15021/healthz/ready": dial tcp 10.133.0.101:15021: connect: connection refused... 50m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f Created pod: payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 50m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w2-b06b08 Scaled up replica set payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f from 0 to 1 50m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c Error creating: pods "payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-b06b08: serviceaccount "payload-processing-e2e-shared-b-w2-b06b08" not found 50m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c Created pod: payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 50m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w2-b06b08 Scaled up replica set payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c from 0 to 1 50m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f Error creating: pods "payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w2-b06b08: serviceaccount "payload-processing-e2e-shared-b-w2-b06b08" not found 49m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 Add eth0 [10.132.0.131/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 Container started 49m Normal Created pod/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 Container created 49m Normal Pulled pod/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 Add eth0 [10.132.0.132/23] from ovn-kubernetes 49m Normal Created pod/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 Container created 49m Normal Started pod/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 Container started 49m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz 49m Normal ScalingReplicaSet deployment/e2e-mt-79f4fb0f-openshift-default Scaled up replica set e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf from 0 to 1 49m Normal AddedLabel gateway/e2e-mt-79f4fb0f Added label istio.io/rev=openshift-gateway to gateway e2e-mt-79f4fb0f 49m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Add eth0 [10.134.0.127/23] from ovn-kubernetes 49m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-79f4fb0f Scaled up replica set payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467 from 0 to 1 49m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467 Created pod: payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 49m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-79f4fb0f Scaled up replica set payload-processing-e2e-mt-79f4fb0f-78c7766bc9 from 0 to 1 49m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467 Error creating: pods "payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-79f4fb0f: serviceaccount "payload-processing-e2e-mt-79f4fb0f" not found 49m Warning FailedCreate replicaset/payload-processing-e2e-mt-79f4fb0f-78c7766bc9 Error creating: pods "payload-processing-e2e-mt-79f4fb0f-78c7766bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-79f4fb0f: serviceaccount "payload-processing-e2e-mt-79f4fb0f" not found 49m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-79f4fb0f-78c7766bc9 Created pod: payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt 49m Normal Created pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Container created 49m Normal AddedInterface pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Add eth0 [10.132.0.133/23] from ovn-kubernetes 49m Normal Started pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Container started 49m Normal Created pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Container created 49m Normal Started pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Container started 49m Normal Pulled pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Normal AddedInterface pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Add eth0 [10.132.0.134/23] from ovn-kubernetes 49m Normal Pulled pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 49m Warning Unhealthy pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Readiness probe failed: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 49m Warning ProbeError pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Readiness probe error: Get "http://10.132.0.126:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-a8c1600c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Pulled pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-a8c1600c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 49m Normal Started pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Container started 49m Normal Created pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Container created 49m Warning Unhealthy pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Startup probe failed: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused 49m Warning ProbeError pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Startup probe error: Get "http://10.132.0.126:15021/healthz/ready": dial tcp 10.132.0.126:15021: connect: connection refused... 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-4f340c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-4f340c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-b06b08-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Started pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Container started 49m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-b06b08-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 49m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Container created 49m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Container started 49m Normal Pulled pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 49m Normal Created pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Container created 49m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Startup probe failed: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused 49m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Startup probe error: Get "http://10.134.0.127:15021/healthz/ready": dial tcp 10.134.0.127:15021: connect: connection refused... 48m Warning Unhealthy pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Startup probe failed: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused 48m Warning ProbeError pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Startup probe error: Get "http://10.132.0.130:15021/healthz/ready": dial tcp 10.132.0.130:15021: connect: connection refused... 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-79f4fb0f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-79f4fb0f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 48m Normal Killing pod/payload-pre-processing-e2e-mt-a8c1600c-764447d479-426vm Stopping container payload-pre-processing 48m Normal Killing pod/payload-processing-e2e-mt-a8c1600c-7d96b7788c-gxm68 Stopping container payload-processing 48m Normal SuccessfulCreate replicaset/e2e-api-a-466002-openshift-default-6b4d4c696f Created pod: e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg 48m Normal ScalingReplicaSet deployment/e2e-api-a-466002-openshift-default Scaled up replica set e2e-api-a-466002-openshift-default-6b4d4c696f from 0 to 1 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w2-4f340c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedInterface pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Add eth0 [10.132.0.137/23] from ovn-kubernetes 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w2-4f340c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal AddedLabel gateway/e2e-api-a-466002 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-466002 48m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-33c10a-openshift-default Scaled up replica set e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-466002 Scaled up replica set payload-processing-e2e-api-a-466002-7d68dc4d5f from 0 to 1 48m Normal AddedLabel gateway/e2e-shared-a-w3-33c10a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w3-33c10a 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-466002-7b88dbfff9 Error creating: pods "payload-pre-processing-e2e-api-a-466002-7b88dbfff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-466002: serviceaccount "payload-processing-e2e-api-a-466002" not found 48m Warning FailedCreate replicaset/payload-processing-e2e-api-a-466002-7d68dc4d5f Error creating: pods "payload-processing-e2e-api-a-466002-7d68dc4d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-466002: serviceaccount "payload-processing-e2e-api-a-466002" not found 48m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc Created pod: e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-466002 Scaled up replica set payload-pre-processing-e2e-api-a-466002-7b88dbfff9 from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-466002-7d68dc4d5f Created pod: payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz 48m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j Add eth0 [10.134.0.129/23] from ovn-kubernetes 48m Normal AddedInterface pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Add eth0 [10.133.0.104/23] from ovn-kubernetes 48m Normal Started pod/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz Container started 48m Normal Created pod/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz Container created 48m Normal Pulled pod/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz Add eth0 [10.132.0.138/23] from ovn-kubernetes 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-466002-7b88dbfff9 Created pod: payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j 48m Normal Pulled pod/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w2-b06b08-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w2-b06b08-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal Started pod/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j Container started 48m Normal Created pod/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j Container created 48m Normal Killing pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Stopping container istio-proxy 48m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85 Error creating: pods "payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-33c10a: serviceaccount "payload-processing-e2e-shared-a-w3-33c10a" not found 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85 Created pod: payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w3-33c10a Scaled up replica set payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85 from 0 to 1 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w3-33c10a Scaled up replica set payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c Created pod: payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c Error creating: pods "payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w3-33c10a: serviceaccount "payload-processing-e2e-shared-a-w3-33c10a" not found 48m Normal Created pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Container created 48m Normal Started pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Container started 48m Normal Created pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Container created 48m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Add eth0 [10.134.0.131/23] from ovn-kubernetes 48m Warning ProbeError pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Readiness probe error: HTTP probe failed with statuscode: 503... 48m Warning Unhealthy pod/e2e-mt-a8c1600c-openshift-default-74b4bf4f85-glvg5 Readiness probe failed: HTTP probe failed with statuscode: 503 48m Normal AddedInterface pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Add eth0 [10.132.0.139/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Container started 48m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-mt-79f4fb0f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-mt-79f4fb0f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 48m Normal ScalingReplicaSet deployment/e2e-api-a-75c956-openshift-default Scaled up replica set e2e-api-a-75c956-openshift-default-6995b9c56c from 0 to 1 48m Normal AddedLabel gateway/e2e-api-a-75c956 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-75c956 48m Normal SuccessfulCreate replicaset/e2e-api-a-75c956-openshift-default-6995b9c56c Created pod: e2e-api-a-75c956-openshift-default-6995b9c56c-hckps 48m Normal AddedInterface pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Add eth0 [10.132.0.140/23] from ovn-kubernetes 48m Warning FailedCreate replicaset/payload-processing-e2e-api-a-75c956-7d588d79fc Error creating: pods "payload-processing-e2e-api-a-75c956-7d588d79fc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-75c956: serviceaccount "payload-processing-e2e-api-a-75c956" not found 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-75c956 Scaled up replica set payload-pre-processing-e2e-api-a-75c956-5ccd94c96c from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c Error creating: pods "payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-75c956: serviceaccount "payload-processing-e2e-api-a-75c956" not found 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c Created pod: payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-75c956 Scaled up replica set payload-processing-e2e-api-a-75c956-7d588d79fc from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-75c956-7d588d79fc Created pod: payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 48m Normal Pulled pod/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Started pod/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 Container started 48m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w Add eth0 [10.132.0.141/23] from ovn-kubernetes 48m Normal Pulled pod/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 Container created 48m Normal Created pod/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w Container created 48m Normal Started pod/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w Container started 48m Normal AddedInterface pod/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 Add eth0 [10.132.0.142/23] from ovn-kubernetes 48m Normal AddedLabel gateway/e2e-api-b-752869 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-752869 48m Normal AddedInterface pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Add eth0 [10.134.0.133/23] from ovn-kubernetes 48m Normal ScalingReplicaSet deployment/e2e-api-b-752869-openshift-default Scaled up replica set e2e-api-b-752869-openshift-default-9cf7bfd89 from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-api-b-752869-openshift-default-9cf7bfd89 Created pod: e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw 48m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-752869-67c9855ff Created pod: payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml 48m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-752869 Scaled up replica set payload-pre-processing-e2e-api-b-752869-67c9855ff from 0 to 1 48m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-752869-67c9855ff Error creating: pods "payload-pre-processing-e2e-api-b-752869-67c9855ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-752869: serviceaccount "payload-processing-e2e-api-b-752869" not found 48m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-752869 Scaled up replica set payload-processing-e2e-api-b-752869-7c98d67ff from 0 to 1 48m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-752869-7c98d67ff Created pod: payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b 48m Warning FailedCreate replicaset/payload-processing-e2e-api-b-752869-7c98d67ff Error creating: pods "payload-processing-e2e-api-b-752869-7c98d67ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-752869: serviceaccount "payload-processing-e2e-api-b-752869" not found 48m Normal AddedInterface pod/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b Add eth0 [10.134.0.135/23] from ovn-kubernetes 48m Normal Pulled pod/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml Add eth0 [10.134.0.134/23] from ovn-kubernetes 48m Normal Created pod/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b Container created 48m Normal Started pod/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml Container started 48m Normal Pulled pod/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 48m Normal Created pod/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml Container created 48m Normal Started pod/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b Container started 48m Normal AddedLabel gateway/e2e-shared-b-w3-abdcb1 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w3-abdcb1 48m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-abdcb1-openshift-default Scaled up replica set e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f from 0 to 1 48m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f Created pod: e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n 48m Normal AddedInterface pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Add eth0 [10.134.0.137/23] from ovn-kubernetes 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w3-abdcb1 Scaled up replica set payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99 Error creating: pods "payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-abdcb1: serviceaccount "payload-processing-e2e-shared-b-w3-abdcb1" not found 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f Error creating: pods "payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w3-abdcb1: serviceaccount "payload-processing-e2e-shared-b-w3-abdcb1" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w3-abdcb1 Scaled up replica set payload-processing-e2e-shared-b-w3-abdcb1-858c897c99 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f Created pod: payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99 Created pod: payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-75c956-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Add eth0 [10.132.0.143/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-75c956-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 47m Normal AddedInterface pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Add eth0 [10.134.0.139/23] from ovn-kubernetes 47m Normal Started pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Container started 47m Normal Created pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Container created 47m Normal Killing pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Stopping container payload-processing 47m Normal Started pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Container started 47m Normal Created pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Container created 47m Normal Pulled pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Killing pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Stopping container payload-pre-processing 47m Normal ScalingReplicaSet deployment/e2e-api-b-cce3e7-openshift-default Scaled up replica set e2e-api-b-cce3e7-openshift-default-7479c8855 from 0 to 1 47m Normal AddedLabel gateway/e2e-api-b-cce3e7 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-cce3e7 47m Normal SuccessfulCreate replicaset/e2e-api-b-cce3e7-openshift-default-7479c8855 Created pod: e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst 47m Normal AddedInterface pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Add eth0 [10.133.0.109/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-cce3e7-555bb8c665 Created pod: payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-cce3e7 Scaled up replica set payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-cce3e7 Scaled up replica set payload-processing-e2e-api-b-cce3e7-555bb8c665 from 0 to 1 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7 Error creating: pods "payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cce3e7: serviceaccount "payload-processing-e2e-api-b-cce3e7" not found 47m Warning FailedCreate replicaset/payload-processing-e2e-api-b-cce3e7-555bb8c665 Error creating: pods "payload-processing-e2e-api-b-cce3e7-555bb8c665-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cce3e7: serviceaccount "payload-processing-e2e-api-b-cce3e7" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7 Created pod: payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 47m Normal Started pod/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 Container started 47m Normal Created pod/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 Container created 47m Normal Pulled pod/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 Add eth0 [10.134.0.140/23] from ovn-kubernetes 47m Normal AddedInterface pod/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs Add eth0 [10.132.0.144/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs Container created 47m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Readiness probe failed: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Normal Started pod/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs Container started 47m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Readiness probe error: Get "http://10.133.0.57:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning ProbeError pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Readiness probe error: Get "http://10.133.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 47m Warning Unhealthy pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Readiness probe failed: Get "http://10.133.0.104:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 47m Warning Unhealthy pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-fb8dt Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 47m Normal AddedLabel gateway/e2e-api-a-580f24 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-580f24 47m Warning Unhealthy pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-mmgr4 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 47m Normal ScalingReplicaSet deployment/e2e-api-a-580f24-openshift-default Scaled up replica set e2e-api-a-580f24-openshift-default-86d4586cbc from 0 to 1 47m Normal SuccessfulCreate replicaset/e2e-api-a-580f24-openshift-default-86d4586cbc Created pod: e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj 47m Normal Pulled pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Pulled pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Created pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Container created 47m Normal AddedInterface pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Add eth0 [10.133.0.110/23] from ovn-kubernetes 47m Normal Created pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Container created 47m Normal Started pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Container started 47m Normal Started pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Container started 47m Normal Created pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Container created 47m Normal Pulled pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 47m Normal Started pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Container started 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-580f24 Scaled up replica set payload-processing-e2e-api-a-580f24-86cfcf9c77 from 0 to 1 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-580f24-86cfcf9c77 Created pod: payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 47m Warning FailedCreate replicaset/payload-processing-e2e-api-a-580f24-86cfcf9c77 Error creating: pods "payload-processing-e2e-api-a-580f24-86cfcf9c77-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-580f24: serviceaccount "payload-processing-e2e-api-a-580f24" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-580f24-85c8f7565f Created pod: payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-580f24-85c8f7565f Error creating: pods "payload-pre-processing-e2e-api-a-580f24-85c8f7565f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-580f24: serviceaccount "payload-processing-e2e-api-a-580f24" not found 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-580f24 Scaled up replica set payload-pre-processing-e2e-api-a-580f24-85c8f7565f from 0 to 1 47m Normal Pulled pod/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Started pod/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 Container started 47m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 Add eth0 [10.134.0.143/23] from ovn-kubernetes 47m Normal Pulled pod/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 Container created 47m Normal AddedInterface pod/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 Add eth0 [10.132.0.147/23] from ovn-kubernetes 47m Warning Unhealthy pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Startup probe failed: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused 47m Warning ProbeError pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Startup probe error: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused... 47m Warning ProbeError pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Startup probe error: Get "http://10.132.0.137:15021/healthz/ready": dial tcp 10.132.0.137:15021: connect: connection refused... 47m Normal Created pod/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 Container created 47m Warning Unhealthy pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Startup probe failed: Get "http://10.132.0.140:15021/healthz/ready": dial tcp 10.132.0.140:15021: connect: connection refused 47m Normal Started pod/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 Container started 47m Warning ProbeError pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Startup probe error: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused... 47m Warning Unhealthy pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Startup probe failed: Get "http://10.133.0.104:15021/healthz/ready": dial tcp 10.133.0.104:15021: connect: connection refused 47m Normal Killing pod/payload-processing-e2e-api-a-466002-7d68dc4d5f-h7pwz Stopping container payload-processing 47m Normal Killing pod/payload-pre-processing-e2e-api-a-466002-7b88dbfff9-sph4j Stopping container payload-pre-processing 47m Normal AddedInterface pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h Add eth0 [10.132.0.149/23] from ovn-kubernetes 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-mt-79f4fb0f-78c7766bc9 Created pod: payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h 47m Normal Pulled pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467 Error creating: pods "payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-79f4fb0f: serviceaccount "payload-processing-e2e-mt-79f4fb0f" not found 47m Warning FailedCreate replicaset/payload-processing-e2e-mt-79f4fb0f-78c7766bc9 Error creating: pods "payload-processing-e2e-mt-79f4fb0f-78c7766bc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-mt-79f4fb0f: serviceaccount "payload-processing-e2e-mt-79f4fb0f" not found 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467 Created pod: payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-mt-79f4fb0f Scaled up replica set payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467 from 0 to 1 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-mt-79f4fb0f Scaled up replica set payload-processing-e2e-mt-79f4fb0f-78c7766bc9 from 0 to 1 47m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-466002-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-466002-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 47m Normal AddedInterface pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 Add eth0 [10.134.0.144/23] from ovn-kubernetes 47m Normal Pulled pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 47m Normal Created pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 Container created 47m Normal Started pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 Container started 47m Normal Created pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h Container created 47m Normal Started pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h Container started 47m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl 47m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-p9ncl Stopping container istio-proxy 47m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Stopping container istio-proxy 47m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled down replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 10 to 8 47m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww 47m Normal ScalingReplicaSet deployment/e2e-api-a-6aa924-openshift-default Scaled up replica set e2e-api-a-6aa924-openshift-default-56b6b4c99 from 0 to 1 47m Normal AddedLabel gateway/e2e-api-a-6aa924 Added label istio.io/rev=openshift-gateway to gateway e2e-api-a-6aa924 47m Normal SuccessfulCreate replicaset/e2e-api-a-6aa924-openshift-default-56b6b4c99 Created pod: e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq 47m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 8; reason: All metrics below target 47m Normal AddedInterface pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Add eth0 [10.133.0.115/23] from ovn-kubernetes 47m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-tdnww Readiness probe error: HTTP probe failed with statuscode: 503... 47m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-a-6aa924 Scaled up replica set payload-pre-processing-e2e-api-a-6aa924-66d4f47767 from 0 to 1 47m Warning FailedCreate replicaset/payload-processing-e2e-api-a-6aa924-775b66d469 Error creating: pods "payload-processing-e2e-api-a-6aa924-775b66d469-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-6aa924: serviceaccount "payload-processing-e2e-api-a-6aa924" not found 47m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-a-6aa924-775b66d469 Created pod: payload-processing-e2e-api-a-6aa924-775b66d469-82hlk 47m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-a-6aa924-66d4f47767 Created pod: payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p 47m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-a-6aa924-66d4f47767 Error creating: pods "payload-pre-processing-e2e-api-a-6aa924-66d4f47767-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-a-6aa924: serviceaccount "payload-processing-e2e-api-a-6aa924" not found 47m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-a-6aa924 Scaled up replica set payload-processing-e2e-api-a-6aa924-775b66d469 from 0 to 1 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p Add eth0 [10.134.0.146/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p Container created 46m Normal Pulled pod/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk Add eth0 [10.134.0.147/23] from ovn-kubernetes 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-466002-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-466002-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Started pod/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p Container started 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-75c956-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal AddedLabel gateway/e2e-shared-a-w6-15dc2f Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w6-15dc2f 46m Normal Pulled pod/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-75c956-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled up replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 0 to 1 46m Normal Created pod/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk Container created 46m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt 46m Normal Started pod/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk Container started 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w3-33c10a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w3-33c10a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Killing pod/payload-processing-e2e-api-a-75c956-7d588d79fc-9kt82 Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-api-a-75c956-5ccd94c96c-ng27w Stopping container payload-pre-processing 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-6aa924-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-6aa924-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Add eth0 [10.134.0.148/23] from ovn-kubernetes 46m Warning ProbeError pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Readiness probe error: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Readiness probe failed: Get "http://10.133.0.109:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal ScalingReplicaSet deployment/e2e-api-b-cbb3d5-openshift-default Scaled up replica set e2e-api-b-cbb3d5-openshift-default-69bf7c9485 from 0 to 1 46m Normal AddedLabel gateway/e2e-api-b-cbb3d5 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-cbb3d5 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-580f24-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-580f24-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal SuccessfulCreate replicaset/e2e-api-b-cbb3d5-openshift-default-69bf7c9485 Created pod: e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns 46m Normal AddedInterface pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Add eth0 [10.132.0.151/23] from ovn-kubernetes 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c Error creating: pods "payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-15dc2f: serviceaccount "payload-processing-e2e-shared-a-w6-15dc2f" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w6-15dc2f Scaled up replica set payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w6-15dc2f Scaled up replica set payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c Created pod: payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr 46m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c Error creating: pods "payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w6-15dc2f: serviceaccount "payload-processing-e2e-shared-a-w6-15dc2f" not found 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c Created pod: payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v 46m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Pulled pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Add eth0 [10.132.0.152/23] from ovn-kubernetes 46m Normal Created pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Container created 46m Normal Started pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Container started 46m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Add eth0 [10.134.0.149/23] from ovn-kubernetes 46m Normal Created pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Container created 46m Warning ProbeError pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Readiness probe error: Get "http://10.134.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 46m Warning Unhealthy pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Readiness probe failed: Get "http://10.134.0.137:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 46m Normal Started pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Container started 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59 Error creating: pods "payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cbb3d5: serviceaccount "payload-processing-e2e-api-b-cbb3d5" not found 46m Normal Started pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Container started 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59 Created pod: payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d 46m Normal Pulled pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-cbb3d5 Scaled up replica set payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59 from 0 to 1 46m Warning FailedCreate replicaset/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f Error creating: pods "payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-cbb3d5: serviceaccount "payload-processing-e2e-api-b-cbb3d5" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f Created pod: payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-cbb3d5 Scaled up replica set payload-processing-e2e-api-b-cbb3d5-dbccc5d5f from 0 to 1 46m Normal Created pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Container created 46m Normal AddedInterface pod/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k Add eth0 [10.134.0.150/23] from ovn-kubernetes 46m Normal Started pod/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k Container started 46m Normal Started pod/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d Container started 46m Normal Created pod/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d Container created 46m Normal Pulled pod/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k Container created 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d Add eth0 [10.132.0.153/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Created pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Container created 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Normal Started pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Container started 46m Normal Created pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Container created 46m Normal Started pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Container started 46m Normal Pulled pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Pulled pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 46m Normal Killing pod/e2e-api-a-466002-openshift-default-6b4d4c696f-f9msg Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5dj6c 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 7; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled down replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 8 to 7 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-cce3e7-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-cce3e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Startup probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Startup probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused 46m Warning ProbeError pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Startup probe error: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Startup probe failed: Get "http://10.134.0.133:15021/healthz/ready": dial tcp 10.134.0.133:15021: connect: connection refused 46m Warning ProbeError pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Startup probe error: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused... 46m Warning Unhealthy pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Startup probe failed: Get "http://10.134.0.137:15021/healthz/ready": dial tcp 10.134.0.137:15021: connect: connection refused 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-abdcb1-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-abdcb1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/payload-pre-processing-e2e-api-b-752869-67c9855ff-xz5ml Stopping container payload-pre-processing 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-752869-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-752869-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal Killing pod/payload-processing-e2e-api-b-752869-7c98d67ff-qnz9b Stopping container payload-processing 46m Normal Killing pod/payload-pre-processing-e2e-mt-79f4fb0f-5fc65ff467-fnt88 Stopping container payload-pre-processing 46m Normal Killing pod/payload-processing-e2e-mt-79f4fb0f-78c7766bc9-89b4h Stopping container payload-processing 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-cce3e7-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-cce3e7-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 46m Normal AddedLabel gateway/e2e-api-b-c45a37 Added label istio.io/rev=openshift-gateway to gateway e2e-api-b-c45a37 46m Normal AddedInterface pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Add eth0 [10.133.0.119/23] from ovn-kubernetes 46m Normal ScalingReplicaSet deployment/e2e-api-b-c45a37-openshift-default Scaled up replica set e2e-api-b-c45a37-openshift-default-84b65d78b5 from 0 to 1 46m Normal SuccessfulCreate replicaset/e2e-api-b-c45a37-openshift-default-84b65d78b5 Created pod: e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh 46m Normal Killing pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Stopping container istio-proxy 46m Warning FailedCreate replicaset/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9 Error creating: pods "payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-c45a37: serviceaccount "payload-processing-e2e-api-b-c45a37" not found 46m Normal ScalingReplicaSet deployment/payload-processing-e2e-api-b-c45a37 Scaled up replica set payload-processing-e2e-api-b-c45a37-58bcf5bf75 from 0 to 1 46m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-api-b-c45a37 Scaled up replica set payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9 from 0 to 1 46m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9 Created pod: payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj 46m Warning FailedCreate replicaset/payload-processing-e2e-api-b-c45a37-58bcf5bf75 Error creating: pods "payload-processing-e2e-api-b-c45a37-58bcf5bf75-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-api-b-c45a37: serviceaccount "payload-processing-e2e-api-b-c45a37" not found 46m Normal SuccessfulCreate replicaset/payload-processing-e2e-api-b-c45a37-58bcf5bf75 Created pod: payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb 46m Normal Created pod/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb Container created 46m Normal AddedInterface pod/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj Add eth0 [10.134.0.155/23] from ovn-kubernetes 46m Normal Pulled pod/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal AddedInterface pod/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb Add eth0 [10.132.0.155/23] from ovn-kubernetes 46m Normal Pulled pod/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 46m Normal Started pod/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb Container started 46m Normal Started pod/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj Container started 46m Normal Created pod/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj Container created 46m Warning Unhealthy pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Readiness probe failed: HTTP probe failed with statuscode: 503 46m Warning ProbeError pod/e2e-api-a-75c956-openshift-default-6995b9c56c-hckps Readiness probe error: HTTP probe failed with statuscode: 503... 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 7; reason: All metrics below target 46m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled down replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 10 to 7 46m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj Stopping container istio-proxy 46m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled down replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 10 to 7 46m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled down replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 10 to 7 46m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 46m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-l52ns 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bz4kj 46m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 7; reason: All metrics below target 46m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 7; reason: All metrics below target 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx 46m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-cbknn Stopping container istio-proxy 46m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt 46m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd 46m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-v8bk2 Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-828jt Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-j5hjx Stopping container istio-proxy 46m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-76c4s Stopping container istio-proxy 46m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-h6l58 Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w3-abdcb1-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 46m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-zc7pd Readiness probe error: HTTP probe failed with statuscode: 503... 46m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w3-abdcb1-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal Killing pod/payload-processing-e2e-api-b-cce3e7-555bb8c665-w28rs Stopping container payload-processing 45m Normal Killing pod/payload-pre-processing-e2e-api-b-cce3e7-86f5d98cb7-rcxn9 Stopping container payload-pre-processing 45m Normal AddedLabel gateway/e2e-shared-b-w6-5e8f29 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w6-5e8f29 45m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9 Created pod: e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb 45m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-5e8f29-openshift-default Scaled up replica set e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9 from 0 to 1 45m Normal AddedInterface pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Add eth0 [10.134.0.156/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-752869-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-752869-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled up replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 1 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Add eth0 [10.132.0.156/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Container created 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Container started 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Add eth0 [10.134.0.157/23] from ovn-kubernetes 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w6-5e8f29 Scaled up replica set payload-processing-e2e-shared-b-w6-5e8f29-94bb49845 from 0 to 1 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Container started 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Container created 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w6-5e8f29 Scaled up replica set payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc from 0 to 1 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845 Created pod: payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc Created pod: payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 45m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845 Error creating: pods "payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-5e8f29: serviceaccount "payload-processing-e2e-shared-b-w6-5e8f29" not found 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc Error creating: pods "payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w6-5e8f29: serviceaccount "payload-processing-e2e-shared-b-w6-5e8f29" not found 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Add eth0 [10.133.0.120/23] from ovn-kubernetes 45m Normal Created pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Container created 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Container started 45m Normal AddedInterface pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Add eth0 [10.134.0.159/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Created pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Container created 45m Normal Started pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Container started 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Container created 45m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Add eth0 [10.134.0.160/23] from ovn-kubernetes 45m Normal Pulled pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Container started 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Container created 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Container started 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Startup probe error: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Startup probe failed: Get "http://10.132.0.156:15021/healthz/ready": dial tcp 10.132.0.156:15021: connect: connection refused 45m Warning ProbeError pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Readiness probe error: Get "http://10.133.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Readiness probe failed: Get "http://10.133.0.115:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Normal Killing pod/e2e-api-b-752869-openshift-default-9cf7bfd89-r49sw Stopping container istio-proxy 45m Normal SuccessfulCreate replicaset/e2e-inf-a-69bdf5-openshift-default-784459fbf9 Created pod: e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x 45m Normal AddedLabel gateway/e2e-inf-a-69bdf5 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-69bdf5 45m Normal ScalingReplicaSet deployment/e2e-inf-a-69bdf5-openshift-default Scaled up replica set e2e-inf-a-69bdf5-openshift-default-784459fbf9 from 0 to 1 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Startup probe failed: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Startup probe error: Get "http://10.134.0.157:15021/healthz/ready": dial tcp 10.134.0.157:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Startup probe failed: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-c45a37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Killing pod/payload-pre-processing-e2e-api-a-580f24-85c8f7565f-zxkv9 Stopping container payload-pre-processing 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Startup probe error: Get "http://10.134.0.148:15021/healthz/ready": dial tcp 10.134.0.148:15021: connect: connection refused... 45m Normal AddedInterface pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Add eth0 [10.134.0.162/23] from ovn-kubernetes 45m Normal Killing pod/payload-processing-e2e-api-a-580f24-86cfcf9c77-ll4l4 Stopping container payload-processing 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-c45a37-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal ScalingReplicaSet deployment/e2e-mt-79f4fb0f-openshift-default Scaled up replica set e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf from 1 to 4 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-79f4fb0f-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Container started 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Container created 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Add eth0 [10.132.0.157/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb 45m Warning Unhealthy pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Readiness probe failed: Get "http://10.133.0.110:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Readiness probe error: Get "http://10.133.0.110:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Add eth0 [10.133.0.121/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Container started 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Container started 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Startup probe failed: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Add eth0 [10.134.0.163/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Container created 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Startup probe error: Get "http://10.133.0.120:15021/healthz/ready": dial tcp 10.133.0.120:15021: connect: connection refused... 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Container created 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-69bdf5-b5b8877df Error creating: pods "payload-processing-e2e-inf-a-69bdf5-b5b8877df-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-69bdf5: serviceaccount "payload-processing-e2e-inf-a-69bdf5" not found 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf Created pod: payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-69bdf5 Scaled up replica set payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-69bdf5-b5b8877df Created pod: payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-69bdf5 Scaled up replica set payload-processing-e2e-inf-a-69bdf5-b5b8877df from 0 to 1 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf Error creating: pods "payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-69bdf5: serviceaccount "payload-processing-e2e-inf-a-69bdf5" not found 45m Normal Created pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Container created 45m Normal Created pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Container created 45m Normal Pulled pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Add eth0 [10.134.0.165/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Started pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Container started 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Add eth0 [10.134.0.164/23] from ovn-kubernetes 45m Normal Started pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Container started 45m Normal Pulled pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Container created 45m Normal Started pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Container started 45m Normal Created pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Container created 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Startup probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused 45m Normal Started pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Container started 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Startup probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Startup probe error: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused... 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 8; reason: All metrics below target 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Startup probe failed: Get "http://10.134.0.163:15021/healthz/ready": dial tcp 10.134.0.163:15021: connect: connection refused 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq 45m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-5m5bq Stopping container istio-proxy 45m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-jhchj Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled down replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 10 to 8 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Startup probe failed: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Startup probe error: Get "http://10.133.0.121:15021/healthz/ready": dial tcp 10.133.0.121:15021: connect: connection refused... 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 9; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hdhld Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled down replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 10 to 9 45m Warning Unhealthy pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Startup probe failed: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused 45m Warning ProbeError pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Startup probe error: Get "http://10.133.0.110:15021/healthz/ready": dial tcp 10.133.0.110:15021: connect: connection refused... 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-mt-79f4fb0f-openshift-default New size: 8; reason: 45m Normal ScalingReplicaSet deployment/e2e-mt-79f4fb0f-openshift-default Scaled up replica set e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf from 4 to 8 45m Warning ProbeError pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Startup probe error: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Startup probe failed: Get "http://10.133.0.115:15021/healthz/ready": dial tcp 10.133.0.115:15021: connect: connection refused 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Container created 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Add eth0 [10.134.0.167/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj 45m Normal SuccessfulCreate replicaset/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf Created pod: e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Container created 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Add eth0 [10.133.0.123/23] from ovn-kubernetes 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Container started 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Container started 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Container created 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Add eth0 [10.132.0.158/23] from ovn-kubernetes 45m Normal AddedInterface pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Add eth0 [10.133.0.122/23] from ovn-kubernetes 45m Normal Pulled pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Container created 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Container started 45m Normal Started pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Container started 45m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Unable to retrieve some image pull secrets (e2e-mt-79f4fb0f-openshift-default-dockercfg-shgv6); attempting to pull the image may not succeed. 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Stopping container istio-proxy 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-7cxj2 Stopping container istio-proxy 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-77cgz Stopping container istio-proxy 45m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Unable to retrieve some image pull secrets (e2e-mt-79f4fb0f-openshift-default-dockercfg-shgv6); attempting to pull the image may not succeed. 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Startup probe error: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Startup probe failed: Get "http://10.134.0.167:15021/healthz/ready": dial tcp 10.134.0.167:15021: connect: connection refused 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-wzxnb Stopping container istio-proxy 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-rk4xj Stopping container istio-proxy 45m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Unable to retrieve some image pull secrets (e2e-mt-79f4fb0f-openshift-default-dockercfg-shgv6); attempting to pull the image may not succeed. 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Startup probe failed: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused 45m Warning FailedToRetrieveImagePullSecret pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Unable to retrieve some image pull secrets (e2e-mt-79f4fb0f-openshift-default-dockercfg-shgv6); attempting to pull the image may not succeed. 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Startup probe error: Get "http://10.132.0.158:15021/healthz/ready": dial tcp 10.132.0.158:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Startup probe error: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-f7n7m Startup probe failed: Get "http://10.133.0.122:15021/healthz/ready": dial tcp 10.133.0.122:15021: connect: connection refused 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Startup probe error: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused... 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Stopping container istio-proxy 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Killing pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-ppwzd Stopping container istio-proxy 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-xr569 Startup probe failed: Get "http://10.133.0.123:15021/healthz/ready": dial tcp 10.133.0.123:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Readiness probe failed: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused 45m Warning ProbeError pod/e2e-mt-79f4fb0f-openshift-default-85bd9f8bdf-q7s8r Readiness probe error: Get "http://10.132.0.157:15021/healthz/ready": dial tcp 10.132.0.157:15021: connect: connection refused... 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x 45m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst 45m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr 45m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 45m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 45m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-49cst Stopping container istio-proxy 45m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj 45m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-zw5sr Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled down replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 9 to 7 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 7; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Stopping container istio-proxy 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 6; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-5dm7x Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled down replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 10 to 7 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 7; reason: All metrics below target 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 7; reason: All metrics below target 45m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9kbfj Stopping container istio-proxy 45m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default Scaled down replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 7 to 6 45m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled down replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 8 to 7 45m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6xmkh Stopping container istio-proxy 45m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rhsf5 Readiness probe error: HTTP probe failed with statuscode: 503... 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-580f24-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-580f24-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Readiness probe error: Get "http://10.134.0.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 45m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Readiness probe failed: Get "http://10.134.0.29:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 45m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-p6wk2 Readiness probe error: Get "http://10.134.0.102:15021/healthz/ready": dial tcp 10.134.0.102:15021: connect: connection refused... 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv 45m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 8; reason: All metrics below target 45m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled up replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 4 to 8 45m Normal Killing pod/payload-processing-e2e-api-a-6aa924-775b66d469-82hlk Stopping container payload-processing 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz 45m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt 45m Normal Killing pod/payload-pre-processing-e2e-api-a-6aa924-66d4f47767-ghf8p Stopping container payload-pre-processing 45m Normal Killing pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Stopping container istio-proxy 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Add eth0 [10.132.0.159/23] from ovn-kubernetes 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Container started 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Add eth0 [10.133.0.124/23] from ovn-kubernetes 45m Normal AddedLabel gateway/e2e-inf-a-f4232c Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-f4232c 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Container created 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Add eth0 [10.134.0.171/23] from ovn-kubernetes 45m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Container started 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Container created 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Container started 45m Normal ScalingReplicaSet deployment/e2e-inf-a-f4232c-openshift-default Scaled up replica set e2e-inf-a-f4232c-openshift-default-8664f8f795 from 0 to 1 45m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Add eth0 [10.133.0.125/23] from ovn-kubernetes 45m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Container started 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Container created 45m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Container created 45m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Add eth0 [10.134.0.172/23] from ovn-kubernetes 45m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-a-6aa924-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-a-6aa924-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 45m Normal Pulled pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 45m Normal Created pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Container created 45m Normal Started pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Container started 45m Warning Unhealthy pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Readiness probe failed: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused 45m Warning ProbeError pod/e2e-api-b-cce3e7-openshift-default-7479c8855-v2jst Readiness probe error: Get "http://10.133.0.109:15021/healthz/ready": dial tcp 10.133.0.109:15021: connect: connection refused... 45m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f Created pod: payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj 45m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f Error creating: pods "payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f4232c: serviceaccount "payload-processing-e2e-inf-a-f4232c" not found 45m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-f4232c Scaled up replica set payload-processing-e2e-inf-a-f4232c-5f5c7fd745 from 0 to 1 45m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-f4232c Scaled up replica set payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f from 0 to 1 45m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-f4232c-5f5c7fd745 Created pod: payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 45m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-f4232c-5f5c7fd745 Error creating: pods "payload-processing-e2e-inf-a-f4232c-5f5c7fd745-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-f4232c: serviceaccount "payload-processing-e2e-inf-a-f4232c" not found 45m Normal ScalingReplicaSet deployment/e2e-inf-a-fb50fa-openshift-default Scaled up replica set e2e-inf-a-fb50fa-openshift-default-5847d9656c from 0 to 1 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Startup probe failed: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Startup probe error: Get "http://10.132.0.159:15021/healthz/ready": dial tcp 10.132.0.159:15021: connect: connection refused... 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Startup probe error: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused... 45m Normal AddedLabel gateway/e2e-inf-a-fb50fa Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-fb50fa 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Startup probe failed: Get "http://10.134.0.171:15021/healthz/ready": dial tcp 10.134.0.171:15021: connect: connection refused 45m Normal SuccessfulCreate replicaset/e2e-inf-a-fb50fa-openshift-default-5847d9656c Created pod: e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Startup probe error: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused... 45m Normal Started pod/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 Container started 45m Normal AddedInterface pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Add eth0 [10.134.0.175/23] from ovn-kubernetes 45m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Add eth0 [10.134.0.173/23] from ovn-kubernetes 45m Normal Pulled pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Normal Pulled pod/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 45m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Startup probe error: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused... 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Startup probe failed: Get "http://10.133.0.124:15021/healthz/ready": dial tcp 10.133.0.124:15021: connect: connection refused 45m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Startup probe failed: Get "http://10.133.0.125:15021/healthz/ready": dial tcp 10.133.0.125:15021: connect: connection refused 45m Normal Created pod/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 Container created 45m Normal Created pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Container created 45m Normal Started pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Container started 45m Normal AddedInterface pod/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 Add eth0 [10.134.0.174/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-inf-a-62778a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-62778a 44m Normal SuccessfulCreate replicaset/e2e-inf-a-62778a-openshift-default-6c966fd5b9 Created pod: e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj 44m Normal ScalingReplicaSet deployment/e2e-inf-a-62778a-openshift-default Scaled up replica set e2e-inf-a-62778a-openshift-default-6c966fd5b9 from 0 to 1 44m Normal AddedInterface pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Add eth0 [10.134.0.176/23] from ovn-kubernetes 44m Warning Unhealthy pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Readiness probe failed: Get "http://10.133.0.119:15021/healthz/ready": read tcp 10.133.0.2:47176->10.133.0.119:15021: read: connection reset by peer 44m Warning Unhealthy pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Startup probe failed: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused 44m Warning ProbeError pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Startup probe error: Get "http://10.132.0.151:15021/healthz/ready": dial tcp 10.132.0.151:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Readiness probe error: Get "http://10.133.0.119:15021/healthz/ready": read tcp 10.133.0.2:47176->10.133.0.119:15021: read: connection reset by peer... 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf Error creating: pods "payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fb50fa: serviceaccount "payload-processing-e2e-inf-a-fb50fa" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf Created pod: payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n 44m Normal Created pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Container created 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9 Created pod: payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-fb50fa Scaled up replica set payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf from 0 to 1 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9 Error creating: pods "payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-fb50fa: serviceaccount "payload-processing-e2e-inf-a-fb50fa" not found 44m Normal Started pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Container started 44m Normal Pulled pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-fb50fa Scaled up replica set payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9 from 0 to 1 44m Normal Created pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Container created 44m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled up replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 8 to 9 44m Normal Started pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Container started 44m Normal SuccessfulCreate replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Created pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-62778a Scaled up replica set payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-62778a Scaled up replica set payload-processing-e2e-inf-a-62778a-649fbc5df5 from 0 to 1 44m Normal Pulled pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Add eth0 [10.134.0.178/23] from ovn-kubernetes 44m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 9; reason: All metrics below target 44m Normal Started pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Container started 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Add eth0 [10.134.0.177/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Container created 44m Normal AddedInterface pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Add eth0 [10.132.0.160/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-62778a-649fbc5df5 Created pod: payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f 44m Normal Started pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Container started 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-62778a-649fbc5df5 Error creating: pods "payload-processing-e2e-inf-a-62778a-649fbc5df5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-62778a: serviceaccount "payload-processing-e2e-inf-a-62778a" not found 44m Normal Pulled pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Container created 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9 Created pod: payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9 Error creating: pods "payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-62778a: serviceaccount "payload-processing-e2e-inf-a-62778a" not found 44m Normal Pulled pod/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Add eth0 [10.134.0.179/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Container started 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f Add eth0 [10.134.0.180/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f Container created 44m Normal Started pod/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f Container started 44m Warning Unhealthy pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Startup probe failed: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused 44m Warning ProbeError pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Startup probe error: Get "http://10.133.0.119:15021/healthz/ready": dial tcp 10.133.0.119:15021: connect: connection refused... 44m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Startup probe error: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Startup probe failed: Get "http://10.132.0.160:15021/healthz/ready": dial tcp 10.132.0.160:15021: connect: connection refused 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-cbb3d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-cbb3d5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Normal SuccessfulCreate replicaset/e2e-inf-b-d4b52a-openshift-default-7d99d6d877 Created pod: e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf 44m Normal AddedLabel gateway/e2e-inf-b-d4b52a Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-d4b52a 44m Normal ScalingReplicaSet deployment/e2e-inf-b-d4b52a-openshift-default Scaled up replica set e2e-inf-b-d4b52a-openshift-default-7d99d6d877 from 0 to 1 44m Normal Killing pod/e2e-api-a-580f24-openshift-default-86d4586cbc-57kvj Stopping container istio-proxy 44m Normal AddedInterface pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Add eth0 [10.134.0.181/23] from ovn-kubernetes 44m Normal AddedLabel gateway/e2e-inf-a-61da9d Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-61da9d 44m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww 44m Normal ScalingReplicaSet deployment/e2e-inf-a-61da9d-openshift-default Scaled up replica set e2e-inf-a-61da9d-openshift-default-58d4d4845c from 0 to 1 44m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Add eth0 [10.134.0.183/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f4232c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f4232c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-d4b52a-69dc48fff5 Error creating: pods "payload-processing-e2e-inf-b-d4b52a-69dc48fff5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d4b52a: serviceaccount "payload-processing-e2e-inf-b-d4b52a" not found 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-d4b52a-69dc48fff5 Created pod: payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9 Error creating: pods "payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-d4b52a: serviceaccount "payload-processing-e2e-inf-b-d4b52a" not found 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-d4b52a Scaled up replica set payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9 from 0 to 1 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-d4b52a Scaled up replica set payload-processing-e2e-inf-b-d4b52a-69dc48fff5 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9 Created pod: payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s 44m Normal Created pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Container created 44m Normal Created pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Container created 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Add eth0 [10.134.0.185/23] from ovn-kubernetes 44m Normal Started pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Container started 44m Normal Pulled pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Add eth0 [10.134.0.186/23] from ovn-kubernetes 44m Normal Started pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Container started 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-61da9d-75c8647c9 Created pod: payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c Error creating: pods "payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-61da9d: serviceaccount "payload-processing-e2e-inf-a-61da9d" not found 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c Created pod: payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-61da9d Scaled up replica set payload-pre-processing-e2e-inf-a-61da9d-657cdf778c from 0 to 1 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-61da9d-75c8647c9 Error creating: pods "payload-processing-e2e-inf-a-61da9d-75c8647c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-61da9d: serviceaccount "payload-processing-e2e-inf-a-61da9d" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-61da9d Scaled up replica set payload-processing-e2e-inf-a-61da9d-75c8647c9 from 0 to 1 44m Normal Created pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Container created 44m Normal AddedInterface pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Add eth0 [10.134.0.188/23] from ovn-kubernetes 44m Normal Created pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Container started 44m Normal Pulled pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Pulled pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Add eth0 [10.134.0.187/23] from ovn-kubernetes 44m Normal Killing pod/payload-processing-e2e-api-b-cbb3d5-dbccc5d5f-z2p7k Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-api-b-cbb3d5-77cb59c59-m7w8d Stopping container payload-pre-processing 44m Normal Started pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Container started 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-cbb3d5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-cbb3d5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal Killing pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Stopping container istio-proxy 44m Warning Unhealthy pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Readiness probe failed: HTTP probe failed with statuscode: 503 44m Warning ProbeError pod/e2e-api-a-6aa924-openshift-default-56b6b4c99-9cxhq Readiness probe error: HTTP probe failed with statuscode: 503... 44m Normal Killing pod/payload-processing-e2e-api-b-c45a37-58bcf5bf75-846fb Stopping container payload-processing 44m Normal Killing pod/payload-pre-processing-e2e-api-b-c45a37-7b55b8cdf9-tzswj Stopping container payload-pre-processing 44m Normal Created pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Container created 44m Normal Pulled pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Pulled pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 44m Normal Created pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Container created 44m Normal Started pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Container started 44m Normal Started pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Container started 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-69bdf5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-69bdf5-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-api-b-c45a37-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-api-b-c45a37-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-5e8f29-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-5e8f29-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 44m Normal SuccessfulCreate replicaset/e2e-inf-b-cae9ec-openshift-default-758c77f6d5 Created pod: e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f 44m Normal ScalingReplicaSet deployment/e2e-inf-b-cae9ec-openshift-default Scaled up replica set e2e-inf-b-cae9ec-openshift-default-758c77f6d5 from 0 to 1 44m Normal AddedLabel gateway/e2e-inf-b-cae9ec Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-cae9ec 44m Normal AddedInterface pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Add eth0 [10.134.0.190/23] from ovn-kubernetes 44m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 44m Warning Unhealthy pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Startup probe failed: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused 44m Warning ProbeError pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Startup probe error: Get "http://10.134.0.162:15021/healthz/ready": dial tcp 10.134.0.162:15021: connect: connection refused... 44m Warning Unhealthy pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Startup probe failed: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused 44m Warning ProbeError pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Startup probe error: Get "http://10.134.0.156:15021/healthz/ready": dial tcp 10.134.0.156:15021: connect: connection refused... 44m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-cae9ec-7667b7b695 Error creating: pods "payload-processing-e2e-inf-b-cae9ec-7667b7b695-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-cae9ec: serviceaccount "payload-processing-e2e-inf-b-cae9ec" not found 44m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-cae9ec Scaled up replica set payload-processing-e2e-inf-b-cae9ec-7667b7b695 from 0 to 1 44m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699 Created pod: payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw 44m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699 Error creating: pods "payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-cae9ec: serviceaccount "payload-processing-e2e-inf-b-cae9ec" not found 44m Normal AddedLabel gateway/e2e-inf-b-61f295 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-61f295 44m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-cae9ec-7667b7b695 Created pod: payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg 44m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-cae9ec Scaled up replica set payload-pre-processing-e2e-inf-b-cae9ec-67f455b699 from 0 to 1 44m Normal Pulled pod/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Add eth0 [10.132.0.164/23] from ovn-kubernetes 44m Normal Pulled pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 44m Normal Created pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Container created 44m Normal Started pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Container started 44m Normal SuccessfulCreate replicaset/e2e-inf-b-61f295-openshift-default-6797665b47 Created pod: e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr 44m Normal Started pod/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg Container started 44m Normal Created pod/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg Container created 44m Normal AddedInterface pod/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg Add eth0 [10.132.0.163/23] from ovn-kubernetes 44m Normal ScalingReplicaSet deployment/e2e-inf-b-61f295-openshift-default Scaled up replica set e2e-inf-b-61f295-openshift-default-6797665b47 from 0 to 1 44m Normal AddedLabel gateway/e2e-inf-b-4fde10 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-4fde10 44m Normal ScalingReplicaSet deployment/e2e-inf-b-4fde10-openshift-default Scaled up replica set e2e-inf-b-4fde10-openshift-default-85c7f6f7f7 from 0 to 1 44m Normal AddedInterface pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Add eth0 [10.133.0.128/23] from ovn-kubernetes 44m Normal SuccessfulCreate replicaset/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7 Created pod: e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm 44m Normal AddedInterface pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Add eth0 [10.133.0.129/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-61f295 Scaled up replica set payload-pre-processing-e2e-inf-b-61f295-865f8d99c5 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5 Created pod: payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-61f295-58c5d7685c Error creating: pods "payload-processing-e2e-inf-b-61f295-58c5d7685c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-61f295: serviceaccount "payload-processing-e2e-inf-b-61f295" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-61f295-58c5d7685c Created pod: payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-61f295 Scaled up replica set payload-processing-e2e-inf-b-61f295-58c5d7685c from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5 Error creating: pods "payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-61f295: serviceaccount "payload-processing-e2e-inf-b-61f295" not found 43m Normal Created pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Container created 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-4fde10 Scaled up replica set payload-pre-processing-e2e-inf-b-4fde10-56488df7f from 0 to 1 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Container started 43m Normal Created pod/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q Container created 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Add eth0 [10.132.0.165/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q Add eth0 [10.132.0.166/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-4fde10 Scaled up replica set payload-processing-e2e-inf-b-4fde10-6b7986f6c from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-4fde10-56488df7f Error creating: pods "payload-pre-processing-e2e-inf-b-4fde10-56488df7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4fde10: serviceaccount "payload-processing-e2e-inf-b-4fde10" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-4fde10-6b7986f6c Error creating: pods "payload-processing-e2e-inf-b-4fde10-6b7986f6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-4fde10: serviceaccount "payload-processing-e2e-inf-b-4fde10" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-4fde10-6b7986f6c Created pod: payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb 43m Normal Created pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Container created 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Add eth0 [10.132.0.167/23] from ovn-kubernetes 43m Normal Started pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Container started 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-4fde10-56488df7f Created pod: payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Add eth0 [10.132.0.168/23] from ovn-kubernetes 43m Normal Pulled pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Container started 43m Normal Created pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Container created 43m Warning Unhealthy pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Readiness probe failed: Get "http://10.134.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Warning ProbeError pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Readiness probe error: Get "http://10.134.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f4232c-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-inf-a-f4232c-openshift-default Scaled up replica set e2e-inf-a-f4232c-openshift-default-8664f8f795 from 1 to 4 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk 43m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Add eth0 [10.133.0.130/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Add eth0 [10.132.0.169/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Add eth0 [10.134.0.191/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Container started 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Container created 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Container created 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Container started 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Container created 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Container started 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Readiness probe error: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Readiness probe failed: Get "http://10.134.0.172:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Pulled pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Container started 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4fde10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4fde10-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Created pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Container created 43m Normal Created pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Container created 43m Normal Pulled pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Container started 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Container started 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fb50fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Container created 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fb50fa-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-69bdf5-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-69bdf5-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Startup probe error: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Startup probe failed: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Startup probe failed: Get "http://10.133.0.130:15021/healthz/ready": dial tcp 10.133.0.130:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Startup probe error: Get "http://10.132.0.169:15021/healthz/ready": dial tcp 10.132.0.169:15021: connect: connection refused... 43m Normal SuccessfulCreate replicaset/e2e-inf-b-0d835b-openshift-default-55fc778f97 Created pod: e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Startup probe error: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Startup probe failed: Get "http://10.134.0.191:15021/healthz/ready": dial tcp 10.134.0.191:15021: connect: connection refused 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w6-5e8f29-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w6-5e8f29-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal AddedLabel gateway/e2e-inf-b-0d835b Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-0d835b 43m Normal ScalingReplicaSet deployment/e2e-inf-b-0d835b-openshift-default Scaled up replica set e2e-inf-b-0d835b-openshift-default-55fc778f97 from 0 to 1 43m Normal Killing pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Stopping container istio-proxy 43m Normal ScalingReplicaSet deployment/e2e-inf-a-1ddade-openshift-default Scaled up replica set e2e-inf-a-1ddade-openshift-default-5df978b75c from 0 to 1 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Startup probe error: Get "http://10.134.0.191:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Startup probe failed: Get "http://10.134.0.191:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal SuccessfulCreate replicaset/e2e-inf-a-1ddade-openshift-default-5df978b75c Created pod: e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx 43m Normal AddedLabel gateway/e2e-inf-a-1ddade Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-1ddade 43m Warning ProbeError pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Readiness probe error: HTTP probe failed with statuscode: 503... 43m Warning Unhealthy pod/e2e-api-b-cbb3d5-openshift-default-69bf7c9485-wvzns Readiness probe failed: HTTP probe failed with statuscode: 503 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-f4232c-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-f4232c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Normal AddedInterface pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Add eth0 [10.134.0.192/23] from ovn-kubernetes 43m Warning Unhealthy pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Startup probe failed: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Startup probe failed: Get "http://10.134.0.175:15021/healthz/ready": dial tcp 10.134.0.175:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Startup probe error: Get "http://10.134.0.175:15021/healthz/ready": dial tcp 10.134.0.175:15021: connect: connection refused... 43m Warning Unhealthy pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Readiness probe failed: timeout: health rpc did not complete within 1s 43m Warning ProbeError pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Startup probe error: Get "http://10.134.0.176:15021/healthz/ready": dial tcp 10.134.0.176:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Startup probe failed: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Startup probe error: Get "http://10.134.0.172:15021/healthz/ready": dial tcp 10.134.0.172:15021: connect: connection refused... 43m Normal AddedInterface pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Add eth0 [10.134.0.194/23] from ovn-kubernetes 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-0d835b Scaled up replica set payload-processing-e2e-inf-b-0d835b-5d8fd5d47f from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-0d835b Scaled up replica set payload-pre-processing-e2e-inf-b-0d835b-545c644787 from 0 to 1 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-0d835b-545c644787 Error creating: pods "payload-pre-processing-e2e-inf-b-0d835b-545c644787-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0d835b: serviceaccount "payload-processing-e2e-inf-b-0d835b" not found 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f Error creating: pods "payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-0d835b: serviceaccount "payload-processing-e2e-inf-b-0d835b" not found 43m Normal Pulled pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Killing pod/e2e-api-b-c45a37-openshift-default-84b65d78b5-4cfhh Stopping container istio-proxy 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f Created pod: payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Add eth0 [10.132.0.171/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-0d835b-545c644787 Created pod: payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-62778a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal Started pod/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r Add eth0 [10.132.0.172/23] from ovn-kubernetes 43m Normal AddedLabel gateway/e2e-inf-a-554759 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-a-554759 43m Normal Created pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Container created 43m Normal Started pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Container started 43m Normal Pulled pod/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Created pod/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r Container created 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-62778a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Normal ScalingReplicaSet deployment/e2e-inf-a-554759-openshift-default Scaled up replica set e2e-inf-a-554759-openshift-default-69c6b6cbdf from 0 to 1 43m Normal SuccessfulCreate replicaset/e2e-inf-a-554759-openshift-default-69c6b6cbdf Created pod: e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-1ddade Scaled up replica set payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595 from 0 to 1 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-1ddade Scaled up replica set payload-processing-e2e-inf-a-1ddade-7c4f444ff9 from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-1ddade-7c4f444ff9 Created pod: payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-1ddade-7c4f444ff9 Error creating: pods "payload-processing-e2e-inf-a-1ddade-7c4f444ff9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1ddade: serviceaccount "payload-processing-e2e-inf-a-1ddade" not found 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595 Error creating: pods "payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-1ddade: serviceaccount "payload-processing-e2e-inf-a-1ddade" not found 43m Normal AddedInterface pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Add eth0 [10.133.0.133/23] from ovn-kubernetes 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595 Created pod: payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 43m Normal Pulled pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Add eth0 [10.134.0.196/23] from ovn-kubernetes 43m Normal Pulled pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Add eth0 [10.132.0.173/23] from ovn-kubernetes 43m Normal Created pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Container created 43m Normal Started pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Container started 43m Normal Created pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Container created 43m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj 43m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r 43m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 43m Normal ScalingReplicaSet deployment/e2e-inf-a-61da9d-openshift-default Scaled up replica set e2e-inf-a-61da9d-openshift-default-58d4d4845c from 1 to 4 43m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Container created 43m Normal ScalingReplicaSet deployment/e2e-inf-a-f4232c-openshift-default Scaled up replica set e2e-inf-a-f4232c-openshift-default-8664f8f795 from 4 to 7 43m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Container started 43m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Container created 43m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Add eth0 [10.133.0.134/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Container started 43m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-f4232c-openshift-default New size: 7; reason: 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p 43m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Add eth0 [10.132.0.175/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Container created 43m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Container started 43m Normal SuccessfulCreate replicaset/e2e-inf-a-f4232c-openshift-default-8664f8f795 Created pod: e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn 43m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Add eth0 [10.132.0.174/23] from ovn-kubernetes 43m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Add eth0 [10.133.0.136/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Add eth0 [10.133.0.135/23] from ovn-kubernetes 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9 Error creating: pods "payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-554759: serviceaccount "payload-processing-e2e-inf-a-554759" not found 43m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-a-554759 Scaled up replica set payload-processing-e2e-inf-a-554759-db5b8c5dc from 0 to 1 43m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9 Created pod: payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd 43m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-a-554759 Scaled up replica set payload-pre-processing-e2e-inf-a-554759-5f8879f7c9 from 0 to 1 43m Normal Pulled pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Normal AddedInterface pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Add eth0 [10.132.0.176/23] from ovn-kubernetes 43m Warning FailedCreate replicaset/payload-processing-e2e-inf-a-554759-db5b8c5dc Error creating: pods "payload-processing-e2e-inf-a-554759-db5b8c5dc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-a-554759: serviceaccount "payload-processing-e2e-inf-a-554759" not found 43m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-a-554759-db5b8c5dc Created pod: payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Container started 43m Normal AddedInterface pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Add eth0 [10.132.0.177/23] from ovn-kubernetes 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Container created 43m Normal Pulled pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Container started 43m Normal AddedInterface pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Add eth0 [10.134.0.197/23] from ovn-kubernetes 43m Normal Started pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Container started 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Container created 43m Normal Created pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Container created 43m Normal Started pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Container started 43m Normal Created pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Container created 43m Normal Pulled pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 43m Normal Started pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Container started 43m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Readiness probe failed: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 43m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 43m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Readiness probe error: Get "http://10.134.0.183:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 43m Normal Created pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Container created 43m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Container started 43m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Container created 43m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Startup probe error: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Startup probe error: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Startup probe error: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Startup probe failed: Get "http://10.132.0.175:15021/healthz/ready": dial tcp 10.132.0.175:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Startup probe failed: Get "http://10.133.0.134:15021/healthz/ready": dial tcp 10.133.0.134:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Startup probe failed: Get "http://10.132.0.174:15021/healthz/ready": dial tcp 10.132.0.174:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-fb50fa-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-fb50fa-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Startup probe error: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Startup probe error: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused... 43m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Startup probe error: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Startup probe failed: Get "http://10.134.0.183:15021/healthz/ready": dial tcp 10.134.0.183:15021: connect: connection refused 43m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Startup probe error: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused... 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Startup probe failed: Get "http://10.133.0.135:15021/healthz/ready": dial tcp 10.133.0.135:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Startup probe failed: Get "http://10.133.0.136:15021/healthz/ready": dial tcp 10.133.0.136:15021: connect: connection refused 43m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Startup probe failed: Get "http://10.132.0.176:15021/healthz/ready": dial tcp 10.132.0.176:15021: connect: connection refused 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-62778a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-62778a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 43m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 42m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff 42m Normal ScalingReplicaSet deployment/e2e-inf-a-61da9d-openshift-default Scaled up replica set e2e-inf-a-61da9d-openshift-default-58d4d4845c from 4 to 8 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default New size: 8; reason: 42m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 42m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx 42m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c (combined from similar events): Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 42m Normal ScalingReplicaSet deployment/e2e-inf-a-61da9d-openshift-default Scaled up replica set e2e-inf-a-61da9d-openshift-default-58d4d4845c from 8 to 10 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-61da9d-openshift-default New size: 10; reason: All metrics below target 42m Warning Unhealthy pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Readiness probe failed: Get "http://10.134.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Readiness probe error: Get "http://10.134.0.181:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Normal SuccessfulCreate replicaset/e2e-inf-a-61da9d-openshift-default-58d4d4845c Created pod: e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl 42m Warning ProbeError pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Readiness probe error: Get "http://10.134.0.190:15021/healthz/ready": read tcp 10.134.0.2:54124->10.134.0.190:15021: read: connection reset by peer... 42m Warning Unhealthy pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Readiness probe failed: Get "http://10.134.0.190:15021/healthz/ready": read tcp 10.134.0.2:54124->10.134.0.190:15021: read: connection reset by peer 42m Normal Pulled pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Created pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Container created 42m Normal Created pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Container created 42m Normal Started pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Container started 42m Normal Started pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Container started 42m Normal SuccessfulCreate replicaset/e2e-inf-a-69bdf5-openshift-default-784459fbf9 Created pod: e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-69bdf5-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 42m Normal ScalingReplicaSet deployment/e2e-inf-a-69bdf5-openshift-default Scaled up replica set e2e-inf-a-69bdf5-openshift-default-784459fbf9 from 1 to 2 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d4b52a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d4b52a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning Unhealthy pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Startup probe failed: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused 42m Warning ProbeError pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Startup probe error: Get "http://10.134.0.190:15021/healthz/ready": dial tcp 10.134.0.190:15021: connect: connection refused... 42m Warning ProbeError pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Startup probe error: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused... 42m Warning Unhealthy pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Startup probe failed: Get "http://10.134.0.181:15021/healthz/ready": dial tcp 10.134.0.181:15021: connect: connection refused 42m Normal AddedLabel gateway/e2e-inf-b-72c210 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-72c210 42m Normal ScalingReplicaSet deployment/e2e-inf-b-72c210-openshift-default Scaled up replica set e2e-inf-b-72c210-openshift-default-5c9fc9cb8c from 0 to 1 42m Normal SuccessfulCreate replicaset/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c Created pod: e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz 42m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-cae9ec-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-cae9ec-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 42m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw 42m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default New size: 5; reason: All metrics below target 42m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Stopping container istio-proxy 42m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Readiness probe failed: HTTP probe failed with statuscode: 503 42m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-kmwbw Readiness probe error: HTTP probe failed with statuscode: 503... 42m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Readiness probe failed: Get "http://10.133.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Readiness probe failed: Get "http://10.133.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Readiness probe error: Get "http://10.133.0.130:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Readiness probe error: Get "http://10.133.0.124:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning ProbeError pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Readiness probe error: Get "http://10.133.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Readiness probe failed: Get "http://10.133.0.125:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Readiness probe error: Get "http://10.134.0.192:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 42m Warning Unhealthy pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Readiness probe failed: Get "http://10.134.0.192:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning Unhealthy pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Readiness probe failed: Get "http://10.133.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 42m Warning ProbeError pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Readiness probe error: Get "http://10.134.0.194:15021/healthz/ready": read tcp 10.134.0.2:51158->10.134.0.194:15021: read: connection reset by peer... 42m Normal Created pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Container created 42m Normal Created pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Container created 42m Normal Started pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Container started 42m Normal Started pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Container started 42m Normal Pulled pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Normal Pulled pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 42m Warning Unhealthy pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Readiness probe failed: Get "http://10.134.0.194:15021/healthz/ready": read tcp 10.134.0.2:51158->10.134.0.194:15021: read: connection reset by peer 42m Warning ProbeError pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Readiness probe error: Get "http://10.133.0.128:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0d835b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0d835b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-d4b52a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-d4b52a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-cae9ec-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-cae9ec-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal SuccessfulCreate replicaset/e2e-inf-b-35b584-openshift-default-556b575787 Created pod: e2e-inf-b-35b584-openshift-default-556b575787-2zj26 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1ddade-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Normal AddedLabel gateway/e2e-inf-b-35b584 Added label istio.io/rev=openshift-gateway to gateway e2e-inf-b-35b584 41m Normal ScalingReplicaSet deployment/e2e-inf-b-35b584-openshift-default Scaled up replica set e2e-inf-b-35b584-openshift-default-556b575787 from 0 to 1 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1ddade-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning Unhealthy pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Startup probe failed: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Startup probe error: Get "http://10.134.0.192:15021/healthz/ready": dial tcp 10.134.0.192:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Startup probe error: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused... 41m Normal Started pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Container started 41m Normal Created pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Container created 41m Warning Unhealthy pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Startup probe failed: Get "http://10.134.0.194:15021/healthz/ready": dial tcp 10.134.0.194:15021: connect: connection refused 41m Normal Pulled pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-61f295-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-61f295-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning ProbeError pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Startup probe error: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Startup probe failed: Get "http://10.133.0.128:15021/healthz/ready": dial tcp 10.133.0.128:15021: connect: connection refused 41m Normal ScalingReplicaSet deployment/e2e-inf-b-cae9ec-openshift-default Scaled up replica set e2e-inf-b-cae9ec-openshift-default-758c77f6d5 from 1 to 2 41m Normal SuccessfulCreate replicaset/e2e-inf-b-cae9ec-openshift-default-758c77f6d5 Created pod: e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-cae9ec-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 41m Warning Unhealthy pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Readiness probe failed: Get "http://10.133.0.133:15021/healthz/ready": read tcp 10.133.0.2:56832->10.133.0.133:15021: read: connection reset by peer 41m Warning ProbeError pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Readiness probe error: Get "http://10.133.0.133:15021/healthz/ready": read tcp 10.133.0.2:56832->10.133.0.133:15021: read: connection reset by peer... 41m Normal Pulled pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Container created 41m Normal Started pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Container started 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 6; reason: All metrics below target 41m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk 41m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs 41m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled down replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 7 to 6 41m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-7bghs Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Stopping container istio-proxy 41m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-pjjfk Stopping container istio-proxy 41m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 6; reason: All metrics below target 41m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled down replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 7 to 6 41m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn 41m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled down replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 7 to 6 41m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Readiness probe error: HTTP probe failed with statuscode: 503... 41m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-8btzn Readiness probe failed: HTTP probe failed with statuscode: 503 41m Warning Unhealthy pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Startup probe failed: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused 41m Warning ProbeError pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Startup probe error: Get "http://10.133.0.133:15021/healthz/ready": dial tcp 10.133.0.133:15021: connect: connection refused... 41m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Add eth0 [10.133.0.145/23] from ovn-kubernetes 41m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Container created 41m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Container started 41m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Add eth0 [10.133.0.146/23] from ovn-kubernetes 41m Normal AddedInterface pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Add eth0 [10.132.0.181/23] from ovn-kubernetes 41m Normal Pulled pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 41m Normal Created pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Container created 41m Normal Started pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Container started 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-0d835b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-0d835b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-554759-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-554759-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-1ddade-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-35b584 Scaled up replica set payload-processing-e2e-inf-b-35b584-5b757cf5d5 from 0 to 1 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-1ddade-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-35b584 Scaled up replica set payload-pre-processing-e2e-inf-b-35b584-655c54f7ff from 0 to 1 41m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff Created pod: payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 41m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff Error creating: pods "payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-35b584: serviceaccount "payload-processing-e2e-inf-b-35b584" not found 41m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-35b584-5b757cf5d5 Error creating: pods "payload-processing-e2e-inf-b-35b584-5b757cf5d5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-35b584: serviceaccount "payload-processing-e2e-inf-b-35b584" not found 41m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-35b584-5b757cf5d5 Created pod: payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 41m Normal Pulled pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal AddedInterface pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Add eth0 [10.132.0.183/23] from ovn-kubernetes 41m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Add eth0 [10.132.0.182/23] from ovn-kubernetes 41m Normal Created pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Container created 41m Normal Started pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Container started 41m Normal Pulled pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 41m Normal Created pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Container created 41m Normal Started pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Container started 41m Warning ProbeError pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Startup probe error: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused... 41m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Startup probe error: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused... 41m Warning Unhealthy pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Startup probe failed: Get "http://10.132.0.181:15021/healthz/ready": dial tcp 10.132.0.181:15021: connect: connection refused 41m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Startup probe failed: Get "http://10.133.0.145:15021/healthz/ready": dial tcp 10.133.0.145:15021: connect: connection refused 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-61f295-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-61f295-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-35b584-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 41m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-35b584-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning ProbeError pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Readiness probe error: Get "http://10.133.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 40m Warning Unhealthy pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Readiness probe failed: Get "http://10.133.0.129:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-a-554759-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-a-554759-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 40m Warning Unhealthy pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Readiness probe failed: timeout: failed to connect service "10.133.0.59:50052" within 1s: context deadline exceeded 40m Normal Pulled pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Created pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Container created 40m Normal Started pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Container started 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 6; reason: All metrics below target 40m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t 40m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-8t82t Stopping container istio-proxy 40m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default Scaled down replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 7 to 6 40m Warning ProbeError pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Startup probe error: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Startup probe failed: Get "http://10.133.0.129:15021/healthz/ready": dial tcp 10.133.0.129:15021: connect: connection refused 40m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Add eth0 [10.132.0.184/23] from ovn-kubernetes 40m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Container started 40m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Container created 40m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Startup probe error: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused... 40m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Startup probe failed: Get "http://10.132.0.184:15021/healthz/ready": dial tcp 10.132.0.184:15021: connect: connection refused 40m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 6; reason: All metrics below target 40m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled down replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 7 to 6 40m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-vcnzc Stopping container istio-proxy 40m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 6; reason: All metrics below target 40m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t Stopping container istio-proxy 40m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-wrn4t 40m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled down replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 7 to 6 40m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-4fde10-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-4fde10-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 40m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Add eth0 [10.132.0.185/23] from ovn-kubernetes 40m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 40m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Container started 40m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Container created 40m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Startup probe failed: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused 40m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Startup probe error: Get "http://10.132.0.185:15021/healthz/ready": dial tcp 10.132.0.185:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Readiness probe error: Get "http://10.133.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 39m Warning Unhealthy pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Readiness probe failed: Get "http://10.133.0.146:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 39m Normal Pulled pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Started pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Container started 39m Normal Created pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Container created 39m Warning Unhealthy pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Startup probe failed: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Startup probe error: Get "http://10.133.0.146:15021/healthz/ready": dial tcp 10.133.0.146:15021: connect: connection refused... 39m Normal SuccessfulCreate replicaset/e2e-inf-a-fb50fa-openshift-default-5847d9656c Created pod: e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-fb50fa-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 39m Normal ScalingReplicaSet deployment/e2e-inf-a-fb50fa-openshift-default Scaled up replica set e2e-inf-a-fb50fa-openshift-default-5847d9656c from 1 to 2 39m Normal Killing pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Deleted pod: maas-default-gateway-openshift-default-5d84bc6b95-vl7kq 39m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 9; reason: All metrics below target 39m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5d84bc6b95 from 10 to 9 39m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled down replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 9 to 7 39m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 7; reason: All metrics below target 39m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv 39m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sclgv Stopping container istio-proxy 39m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g Stopping container istio-proxy 39m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-sxr7g 39m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Readiness probe failed: HTTP probe failed with statuscode: 503 39m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-vl7kq Readiness probe error: HTTP probe failed with statuscode: 503... 39m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Container started 39m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Container created 39m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Add eth0 [10.132.0.187/23] from ovn-kubernetes 39m Normal Started pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Container started 39m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Container started 39m Normal Created pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Container created 39m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Container created 39m Normal Pulled pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 39m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Add eth0 [10.132.0.188/23] from ovn-kubernetes 39m Normal AddedInterface pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Add eth0 [10.133.0.149/23] from ovn-kubernetes 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-72c210-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 39m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Startup probe failed: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-35b584-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Startup probe failed: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused 39m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-inf-b-35b584-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 39m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Startup probe error: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused... 39m Warning Unhealthy pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Startup probe failed: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused 39m Warning ProbeError pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Startup probe error: Get "http://10.133.0.149:15021/healthz/ready": dial tcp 10.133.0.149:15021: connect: connection refused... 39m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Startup probe error: Get "http://10.132.0.188:15021/healthz/ready": dial tcp 10.132.0.188:15021: connect: connection refused... 39m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-72c210-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 38m Normal ScalingReplicaSet deployment/e2e-inf-b-d4b52a-openshift-default Scaled up replica set e2e-inf-b-d4b52a-openshift-default-7d99d6d877 from 1 to 2 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-d4b52a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 38m Normal SuccessfulCreate replicaset/e2e-inf-b-d4b52a-openshift-default-7d99d6d877 Created pod: e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Deleted pod: maas-default-gateway-openshift-default-5d84bc6b95-vcml7 38m Normal Killing pod/maas-default-gateway-openshift-default-5d84bc6b95-vcml7 Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5d84bc6b95 from 9 to 8 38m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 6; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled down replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 7 to 6 38m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g9lwt 38m Normal Killing pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Stopping container payload-processing 38m Normal Killing pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Stopping container payload-pre-processing 38m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default Scaled down replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 6 to 5 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 5; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-btf5d 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default New size: 5; reason: All metrics below target 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 5; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default Scaled down replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 6 to 5 38m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7vfs5 38m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rkm66 Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default Scaled down replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 6 to 5 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled down replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 6 to 5 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 5; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-r6sfw 38m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-5qw7j 38m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-dkvh6 Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 5; reason: All metrics below target 38m Normal AddedInterface pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Add eth0 [10.134.0.198/23] from ovn-kubernetes 38m Normal Created pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Container created 38m Normal Started pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Container started 38m Normal Pulled pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Add eth0 [10.134.0.199/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal Created pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Container created 38m Normal Started pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Container started 38m Normal AddedInterface pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Add eth0 [10.132.0.190/23] from ovn-kubernetes 38m Normal Pulled pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Add eth0 [10.132.0.189/23] from ovn-kubernetes 38m Normal Started pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Container started 38m Normal Created pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Container created 38m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Startup probe failed: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Startup probe error: Get "http://10.134.0.199:15021/healthz/ready": dial tcp 10.134.0.199:15021: connect: connection refused... 38m Warning ProbeError pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Startup probe error: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused... 38m Warning Unhealthy pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Startup probe failed: Get "http://10.134.0.198:15021/healthz/ready": dial tcp 10.134.0.198:15021: connect: connection refused 38m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5d84bc6b95 from 8 to 7 38m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 7; reason: All metrics below target 38m Normal ScalingReplicaSet deployment/payload-processing-e2e-inf-b-72c210 Scaled up replica set payload-processing-e2e-inf-b-72c210-68bd768b5 from 0 to 1 38m Warning FailedCreate replicaset/payload-processing-e2e-inf-b-72c210-68bd768b5 Error creating: pods "payload-processing-e2e-inf-b-72c210-68bd768b5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-72c210: serviceaccount "payload-processing-e2e-inf-b-72c210" not found 38m Warning FailedCreate replicaset/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59 Error creating: pods "payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-inf-b-72c210: serviceaccount "payload-processing-e2e-inf-b-72c210" not found 38m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Deleted pod: maas-default-gateway-openshift-default-5d84bc6b95-5c26q 38m Normal SuccessfulCreate replicaset/payload-processing-e2e-inf-b-72c210-68bd768b5 Created pod: payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs 38m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59 Created pod: payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 38m Normal Killing pod/maas-default-gateway-openshift-default-5d84bc6b95-5c26q Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-inf-b-72c210 Scaled up replica set payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59 from 0 to 1 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 5; reason: All metrics below target 38m Normal Started pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Container started 38m Normal AddedInterface pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Add eth0 [10.134.0.201/23] from ovn-kubernetes 38m Normal Created pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Container created 38m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled down replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 6 to 5 38m Normal Started pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Container started 38m Normal Pulled pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal Created pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Container created 38m Normal Pulled pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 38m Normal AddedInterface pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Add eth0 [10.134.0.200/23] from ovn-kubernetes 38m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-q7zxg 38m Warning Unhealthy pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Startup probe failed: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused 38m Warning ProbeError pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Startup probe error: Get "http://10.132.0.189:15021/healthz/ready": dial tcp 10.132.0.189:15021: connect: connection refused... 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default New size: 4; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default New size: 4; reason: All metrics below target 38m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-69bdf5-849f4d78cf-shr6d Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 38m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl 38m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-xs6s9 38m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default Scaled down replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 6 to 4 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 4; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb Stopping container istio-proxy 38m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-gv9wb 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-45dcg 38m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 38m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Readiness probe error: HTTP probe failed with statuscode: 503... 38m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt 38m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Readiness probe failed: HTTP probe failed with statuscode: 503 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default New size: 4; reason: All metrics below target 38m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 38m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-mqrn8 Stopping container istio-proxy 38m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 4; reason: All metrics below target 38m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-8kpn6 Stopping container istio-proxy 38m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default Scaled down replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 5 to 4 38m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-sbnsl Stopping container istio-proxy 38m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-l55lt Stopping container istio-proxy 37m Warning Unhealthy pod/payload-processing-e2e-inf-a-69bdf5-b5b8877df-knk8s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal Killing pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-5rz22 Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-a-69bdf5-openshift-default-784459fbf9-td25x Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Deleted pod: maas-default-gateway-openshift-default-5d84bc6b95-srpzw 37m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5d84bc6b95 from 7 to 6 37m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 6; reason: All metrics below target 37m Normal Killing pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Stopping container istio-proxy 37m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz Stopping container istio-proxy 37m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Readiness probe failed: HTTP probe failed with statuscode: 503 37m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-srpzw Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled down replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 5 to 4 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 4; reason: All metrics below target 37m Normal Killing pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Stopping container payload-processing 37m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-dghcz 37m Normal Killing pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Stopping container payload-pre-processing 37m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb Stopping container istio-proxy 37m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-mlgfb 37m Warning Unhealthy pod/payload-processing-e2e-inf-b-d4b52a-69dc48fff5-cmz2p Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-d4b52a-5f746bd8c9-qqs6s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 37m Normal Killing pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Stopping container istio-proxy 37m Normal Killing pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-qpnrf Stopping container istio-proxy 37m Normal AddedLabel gateway/e2e-shared-a-w0-6a3533 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w0-6a3533 37m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9 Created pod: e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx 37m Warning Unhealthy pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Readiness probe failed: HTTP probe failed with statuscode: 503 37m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-6a3533-openshift-default Scaled up replica set e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9 from 0 to 1 37m Warning ProbeError pod/e2e-inf-b-d4b52a-openshift-default-7d99d6d877-l6g6q Readiness probe error: HTTP probe failed with statuscode: 503... 37m Normal AddedInterface pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Add eth0 [10.133.0.159/23] from ovn-kubernetes 37m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r Stopping container istio-proxy 37m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default New size: 3; reason: All metrics below target 37m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w0-6a3533 Scaled up replica set payload-processing-e2e-shared-a-w0-6a3533-68fb85b995 from 0 to 1 37m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-c4h9r 37m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w0-6a3533 Scaled up replica set payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc from 0 to 1 37m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc Error creating: pods "payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-6a3533: serviceaccount "payload-processing-e2e-shared-a-w0-6a3533" not found 37m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995 Error creating: pods "payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w0-6a3533: serviceaccount "payload-processing-e2e-shared-a-w0-6a3533" not found 37m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995 Created pod: payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 37m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc Created pod: payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj 37m Normal Started pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Container started 37m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Add eth0 [10.134.0.214/23] from ovn-kubernetes 37m Normal Pulled pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 37m Normal Created pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Container created 37m Normal Started pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Container started 37m Normal Created pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Container created 37m Normal AddedInterface pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Add eth0 [10.134.0.215/23] from ovn-kubernetes 37m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 36m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: All metrics below target 36m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-wb6gv 36m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 4 to 3 36m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-wb6gv Stopping container discovery 36m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt 36m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-hq6tt Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 36m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 36m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Stopping container payload-pre-processing 36m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd 36m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j Stopping container istio-proxy 36m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default New size: 3; reason: All metrics below target 36m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-7n9t5 Stopping container istio-proxy 36m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-9g96j 36m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4hwd9 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-mqwqd Stopping container istio-proxy 36m Normal Killing pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Stopping container payload-processing 36m Normal Killing pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Stopping container istio-proxy 36m Warning Unhealthy pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Readiness probe failed: HTTP probe failed with statuscode: 503 36m Normal Killing pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-67k5b Stopping container istio-proxy 36m Warning ProbeError pod/e2e-inf-a-fb50fa-openshift-default-5847d9656c-f7dt7 Readiness probe error: HTTP probe failed with statuscode: 503... 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-fb50fa-c8f6d95f9-sn2vg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning Unhealthy pod/payload-processing-e2e-inf-a-fb50fa-5cdd7d8ddf-cnw9n Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Normal Killing pod/payload-processing-e2e-inf-b-61f295-58c5d7685c-2mf2q Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Stopping container payload-pre-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-inf-a-f4232c-5f5c7fd745-g7pp8 Stopping container payload-processing 36m Normal Killing pod/payload-processing-e2e-inf-a-62778a-649fbc5df5-49w6f Stopping container payload-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Stopping container payload-pre-processing 36m Normal Killing pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Stopping container payload-pre-processing 36m Normal Killing pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Stopping container payload-processing 36m Warning ProbeError pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Readiness probe error: Get "http://10.132.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 36m Warning Unhealthy pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Readiness probe failed: Get "http://10.132.0.190:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 36m Normal Started pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Container started 36m Normal Created pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Container created 36m Normal Pulled pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 36m Normal Killing pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Stopping container istio-proxy 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-61f295-865f8d99c5-zjwcs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 36m Warning ProbeError pod/e2e-inf-a-62778a-openshift-default-6c966fd5b9-s9gxj Readiness probe error: HTTP probe failed with statuscode: 503... 36m Normal AddedLabel gateway/e2e-shared-b-w0-612b46 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w0-612b46 36m Warning Unhealthy pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Startup probe failed: Get "http://10.132.0.190:15021/healthz/ready": dial tcp 10.132.0.190:15021: connect: connection refused 36m Normal ScalingReplicaSet deployment/e2e-shared-b-w0-612b46-openshift-default Scaled up replica set e2e-shared-b-w0-612b46-openshift-default-79fc6676c5 from 0 to 1 36m Normal SuccessfulCreate replicaset/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5 Created pod: e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb 36m Warning ProbeError pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Startup probe error: Get "http://10.132.0.190:15021/healthz/ready": dial tcp 10.132.0.190:15021: connect: connection refused... 36m Normal AddedInterface pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Add eth0 [10.134.0.225/23] from ovn-kubernetes 36m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-62778a-6cd8c4dfc9-xfrs7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f Error creating: pods "payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-612b46: serviceaccount "payload-processing-e2e-shared-b-w0-612b46" not found 35m Normal Killing pod/e2e-inf-b-61f295-openshift-default-6797665b47-ws5wr Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w0-612b46 Scaled up replica set payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w0-612b46 Scaled up replica set payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f Created pod: payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9 Created pod: payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9 Error creating: pods "payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w0-612b46: serviceaccount "payload-processing-e2e-shared-b-w0-612b46" not found 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-f4232c-575c7fd47f-sblsj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-inf-b-72c210-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Normal Started pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Container started 35m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Container created 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Add eth0 [10.134.0.227/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Container created 35m Normal Started pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Container started 35m Normal AddedInterface pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Add eth0 [10.134.0.226/23] from ovn-kubernetes 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Stopping container payload-processing 35m Warning Unhealthy pod/payload-processing-e2e-inf-a-61da9d-75c8647c9-b7t8t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-61da9d-657cdf778c-cr5gl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-hgz4n Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-ggv6k Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-t9mj4 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-kw4bj Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-j249r Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-h52p6 Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-pm4nr Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-sjmnl Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-fjvjh Stopping container istio-proxy 35m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-7ntgk Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-zmsff Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Stopping container istio-proxy 35m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-c28ww Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jwg92 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-jvt9p Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning ProbeError pod/e2e-inf-a-f4232c-openshift-default-8664f8f795-4fgqn Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Readiness probe failed: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused 35m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-s9zlx Readiness probe error: Get "http://10.132.0.187:15021/healthz/ready": dial tcp 10.132.0.187:15021: connect: connection refused... 35m Warning ProbeError pod/e2e-inf-a-61da9d-openshift-default-58d4d4845c-bf8c6 Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/payload-processing-e2e-inf-b-cae9ec-7667b7b695-b2lfg Stopping container payload-processing 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Stopping container payload-pre-processing 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-4fde10-56488df7f-htwcf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning Unhealthy pod/payload-processing-e2e-inf-b-4fde10-6b7986f6c-9tvcb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Killing pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Stopping container payload-pre-processing 35m Normal Killing pod/payload-processing-e2e-inf-b-0d835b-5d8fd5d47f-9lk7r Stopping container payload-processing 35m Normal Killing pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Stopping container istio-proxy 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c Created pod: e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 35m Normal AddedLabel gateway/e2e-shared-a-w1-97927b Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w1-97927b 35m Normal AddedInterface pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Add eth0 [10.134.0.231/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-97927b-openshift-default Scaled up replica set e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c from 0 to 1 35m Warning ProbeError pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-b-4fde10-openshift-default-85c7f6f7f7-7zmjm Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w1-97927b Scaled up replica set payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95 from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95 Created pod: payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9 Error creating: pods "payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-97927b: serviceaccount "payload-processing-e2e-shared-a-w1-97927b" not found 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95 Error creating: pods "payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w1-97927b: serviceaccount "payload-processing-e2e-shared-a-w1-97927b" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9 Created pod: payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w1-97927b Scaled up replica set payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9 from 0 to 1 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Add eth0 [10.132.0.211/23] from ovn-kubernetes 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Add eth0 [10.132.0.210/23] from ovn-kubernetes 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Container started 35m Normal Pulled pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Container created 35m Normal Started pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Container created 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-0d835b-545c644787-crtpl Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Normal Killing pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Stopping container istio-proxy 35m Normal Killing pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-j8vdf Stopping container istio-proxy 35m Normal AddedLabel gateway/e2e-shared-a-w4-a39e7e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w4-a39e7e 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-a39e7e-openshift-default Scaled up replica set e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c Created pod: e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh 35m Normal AddedInterface pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Add eth0 [10.134.0.232/23] from ovn-kubernetes 35m Warning Unhealthy pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Readiness probe failed: HTTP probe failed with statuscode: 503 35m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-cae9ec-67f455b699-d7jmw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 35m Warning ProbeError pod/e2e-inf-b-cae9ec-openshift-default-758c77f6d5-g2c5f Readiness probe error: HTTP probe failed with statuscode: 503... 35m Normal Killing pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Stopping container istio-proxy 35m Normal ScalingReplicaSet deployment/e2e-shared-a-w5-974711-openshift-default Scaled up replica set e2e-shared-a-w5-974711-openshift-default-797ffd4df5 from 0 to 1 35m Normal SuccessfulCreate replicaset/e2e-shared-a-w5-974711-openshift-default-797ffd4df5 Created pod: e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 35m Normal AddedLabel gateway/e2e-shared-a-w5-974711 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-a-w5-974711 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w4-a39e7e Scaled up replica set payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf from 0 to 1 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf Created pod: payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c Created pod: payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf Error creating: pods "payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-a39e7e: serviceaccount "payload-processing-e2e-shared-a-w4-a39e7e" not found 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c Error creating: pods "payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w4-a39e7e: serviceaccount "payload-processing-e2e-shared-a-w4-a39e7e" not found 35m Normal AddedInterface pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Add eth0 [10.132.0.212/23] from ovn-kubernetes 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w4-a39e7e Scaled up replica set payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c from 0 to 1 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Container created 35m Normal Created pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Container created 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Container started 35m Normal Started pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Container started 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Add eth0 [10.132.0.213/23] from ovn-kubernetes 35m Normal Pulled pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Add eth0 [10.134.0.234/23] from ovn-kubernetes 35m Normal Pulled pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 35m Normal Started pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Container started 35m Warning ProbeError pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Readiness probe error: HTTP probe failed with statuscode: 503... 35m Warning Unhealthy pod/e2e-inf-b-0d835b-openshift-default-55fc778f97-ljrdp Readiness probe failed: HTTP probe failed with statuscode: 503 35m Normal Created pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Container created 35m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-a-w5-974711-5c9c544565 Created pod: payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs 35m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9 Error creating: pods "payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-974711: serviceaccount "payload-processing-e2e-shared-a-w5-974711" not found 35m Warning FailedCreate replicaset/payload-processing-e2e-shared-a-w5-974711-5c9c544565 Error creating: pods "payload-processing-e2e-shared-a-w5-974711-5c9c544565-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-a-w5-974711: serviceaccount "payload-processing-e2e-shared-a-w5-974711" not found 35m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9 Created pod: payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk 35m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-a-w5-974711 Scaled up replica set payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9 from 0 to 1 35m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-a-w5-974711 Scaled up replica set payload-processing-e2e-shared-a-w5-974711-5c9c544565 from 0 to 1 35m Normal Pulled pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Add eth0 [10.132.0.214/23] from ovn-kubernetes 35m Normal Started pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Container started 35m Normal Created pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Container created 35m Normal Started pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Container started 35m Normal Created pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Container created 35m Normal Pulled pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 35m Normal AddedInterface pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Add eth0 [10.134.0.236/23] from ovn-kubernetes 35m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-97927b-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-97927b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 35m Warning Unhealthy pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Startup probe failed: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused 35m Warning ProbeError pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Startup probe error: Get "http://10.133.0.159:15021/healthz/ready": dial tcp 10.133.0.159:15021: connect: connection refused... 34m Normal AddedLabel gateway/e2e-shared-b-w1-599291 Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w1-599291 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-599291-openshift-default-69f444c65c Created pod: e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-599291-openshift-default Scaled up replica set e2e-shared-b-w1-599291-openshift-default-69f444c65c from 0 to 1 34m Normal AddedInterface pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Add eth0 [10.134.0.238/23] from ovn-kubernetes 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w0-6a3533-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w0-6a3533-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9 Error creating: pods "payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-599291: serviceaccount "payload-processing-e2e-shared-b-w1-599291" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w1-599291-68bc448d7 Created pod: payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Add eth0 [10.132.0.217/23] from ovn-kubernetes 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w1-599291 Scaled up replica set payload-processing-e2e-shared-b-w1-599291-68bc448d7 from 0 to 1 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w1-599291-68bc448d7 Error creating: pods "payload-processing-e2e-shared-b-w1-599291-68bc448d7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w1-599291: serviceaccount "payload-processing-e2e-shared-b-w1-599291" not found 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9 Created pod: payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w1-599291 Scaled up replica set payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9 from 0 to 1 34m Normal Started pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Container started 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Container created 34m Normal Created pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Container created 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Container started 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Add eth0 [10.134.0.239/23] from ovn-kubernetes 34m Normal Pulled pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedLabel gateway/e2e-shared-b-w4-35d13e Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w4-35d13e 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-35d13e-openshift-default Scaled up replica set e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99 Created pod: e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s 34m Normal AddedInterface pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Add eth0 [10.132.0.218/23] from ovn-kubernetes 34m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-14ba5a-openshift-default Scaled up replica set e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f from 0 to 1 34m Normal AddedLabel gateway/e2e-shared-b-w5-14ba5a Added label istio.io/rev=openshift-gateway to gateway e2e-shared-b-w5-14ba5a 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w4-35d13e Scaled up replica set payload-processing-e2e-shared-b-w4-35d13e-5857c666d9 from 0 to 1 34m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f Created pod: e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w4-35d13e Scaled up replica set payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c from 0 to 1 34m Normal AddedInterface pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Add eth0 [10.132.0.219/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c Created pod: payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9 Created pod: payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c Error creating: pods "payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-35d13e: serviceaccount "payload-processing-e2e-shared-b-w4-35d13e" not found 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9 Error creating: pods "payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w4-35d13e: serviceaccount "payload-processing-e2e-shared-b-w4-35d13e" not found 34m Normal Pulled pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Add eth0 [10.134.0.241/23] from ovn-kubernetes 34m Normal Created pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Container created 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Container started 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Container created 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Add eth0 [10.134.0.240/23] from ovn-kubernetes 34m Normal Started pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Container started 34m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-shared-b-w5-14ba5a Scaled up replica set payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c from 0 to 1 34m Warning FailedCreate replicaset/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c Error creating: pods "payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-14ba5a: serviceaccount "payload-processing-e2e-shared-b-w5-14ba5a" not found 34m Normal Started pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Container started 34m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c Created pod: payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f 34m Normal Created pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Container created 34m Warning FailedCreate replicaset/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69 Error creating: pods "payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-shared-b-w5-14ba5a: serviceaccount "payload-processing-e2e-shared-b-w5-14ba5a" not found 34m Normal SuccessfulCreate replicaset/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69 Created pod: payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 34m Normal ScalingReplicaSet deployment/payload-processing-e2e-shared-b-w5-14ba5a Scaled up replica set payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69 from 0 to 1 34m Normal Pulled pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning Unhealthy pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Readiness probe failed: Get "http://10.134.0.231:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 34m Warning ProbeError pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Readiness probe error: Get "http://10.134.0.231:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 34m Normal Pulled pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Started pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Container started 34m Normal Created pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Container created 34m Normal AddedInterface pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Add eth0 [10.134.0.243/23] from ovn-kubernetes 34m Normal AddedInterface pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Add eth0 [10.134.0.242/23] from ovn-kubernetes 34m Normal Pulled pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 34m Normal Created pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Container created 34m Normal Started pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Container started 34m Normal Started pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Container started 34m Normal Created pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Container created 34m Normal Pulled pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-974711-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-b-35b584-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal ScalingReplicaSet deployment/e2e-inf-a-1ddade-openshift-default Scaled up replica set e2e-inf-a-1ddade-openshift-default-5df978b75c from 1 to 2 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-inf-a-1ddade-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-974711-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 34m Normal Pulled pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Started pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Container started 34m Normal AddedInterface pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Add eth0 [10.134.0.244/23] from ovn-kubernetes 34m Normal SuccessfulCreate replicaset/e2e-inf-b-35b584-openshift-default-556b575787 Created pod: e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 34m Normal Created pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Container created 34m Normal Started pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Container started 34m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9 Created pod: e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g 34m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-6a3533-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 34m Normal ScalingReplicaSet deployment/e2e-inf-b-35b584-openshift-default Scaled up replica set e2e-inf-b-35b584-openshift-default-556b575787 from 1 to 2 34m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-6a3533-openshift-default Scaled up replica set e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9 from 1 to 2 34m Normal SuccessfulCreate replicaset/e2e-inf-a-1ddade-openshift-default-5df978b75c Created pod: e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn 34m Normal AddedInterface pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Add eth0 [10.133.0.162/23] from ovn-kubernetes 34m Normal Pulled pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Normal Created pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Container created 34m Normal AddedInterface pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Add eth0 [10.132.0.220/23] from ovn-kubernetes 34m Normal Started pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Container started 34m Normal Pulled pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 34m Warning Unhealthy pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Startup probe failed: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Startup probe error: Get "http://10.132.0.212:15021/healthz/ready": dial tcp 10.132.0.212:15021: connect: connection refused... 34m Normal Created pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Container created 34m Warning Unhealthy pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Startup probe failed: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused 34m Warning ProbeError pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Startup probe error: Get "http://10.134.0.231:15021/healthz/ready": dial tcp 10.134.0.231:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Startup probe failed: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused 34m Warning ProbeError pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Startup probe error: Get "http://10.133.0.162:15021/healthz/ready": dial tcp 10.133.0.162:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Startup probe error: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused... 34m Warning ProbeError pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Startup probe error: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused... 34m Warning Unhealthy pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Startup probe failed: Get "http://10.132.0.220:15021/healthz/ready": dial tcp 10.132.0.220:15021: connect: connection refused 34m Warning Unhealthy pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Startup probe failed: Get "http://10.134.0.244:15021/healthz/ready": dial tcp 10.134.0.244:15021: connect: connection refused 33m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w6-5e8f29-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 33m Normal AddedInterface pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Add eth0 [10.133.0.163/23] from ovn-kubernetes 33m Normal ScalingReplicaSet deployment/e2e-shared-b-w6-5e8f29-openshift-default Scaled up replica set e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9 from 1 to 2 33m Normal SuccessfulCreate replicaset/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9 Created pod: e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv 33m Normal Pulled pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Container started 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w5-974711-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Normal Created pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Container created 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w5-974711-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning Unhealthy pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Startup probe failed: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Startup probe error: Get "http://10.133.0.163:15021/healthz/ready": dial tcp 10.133.0.163:15021: connect: connection refused... 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w1-97927b-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w1-97927b-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning ProbeError pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Readiness probe error: Get "http://10.132.0.218:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Warning Unhealthy pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Readiness probe failed: Get "http://10.132.0.218:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Readiness probe failed: Get "http://10.132.0.219:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 33m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Readiness probe error: Get "http://10.132.0.219:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 33m Normal Created pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Container created 33m Warning Unhealthy pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Readiness probe failed: Get "http://10.134.0.225:15021/healthz/ready": read tcp 10.134.0.2:60896->10.134.0.225:15021: read: connection reset by peer 33m Normal Created pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Container created 33m Normal Started pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Container started 33m Normal Pulled pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Container started 33m Normal Created pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Container created 33m Normal Pulled pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Warning ProbeError pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Readiness probe error: Get "http://10.134.0.225:15021/healthz/ready": read tcp 10.134.0.2:60896->10.134.0.225:15021: read: connection reset by peer... 33m Normal Pulled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Started pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Container started 33m Normal Started pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Container started 33m Normal Pulled pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 33m Normal Created pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Container created 33m Warning Unhealthy pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Startup probe failed: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Startup probe failed: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Startup probe error: Get "http://10.132.0.219:15021/healthz/ready": dial tcp 10.132.0.219:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Startup probe error: Get "http://10.132.0.218:15021/healthz/ready": dial tcp 10.132.0.218:15021: connect: connection refused... 33m Warning ProbeError pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Startup probe error: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused... 33m Warning Unhealthy pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Startup probe failed: Get "http://10.134.0.225:15021/healthz/ready": dial tcp 10.134.0.225:15021: connect: connection refused 33m Warning Unhealthy pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Startup probe failed: Get "http://10.134.0.238:15021/healthz/ready": dial tcp 10.134.0.238:15021: connect: connection refused 33m Warning ProbeError pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Startup probe error: Get "http://10.134.0.238:15021/healthz/ready": dial tcp 10.134.0.238:15021: connect: connection refused... 33m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-612b46-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 33m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-612b46-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w4-35d13e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w4-35d13e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w0-612b46-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w0-612b46-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 32m Normal Killing pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Stopping container payload-pre-processing 32m Normal Killing pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Stopping container payload-processing 32m Normal Started pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Container started 32m Normal Created pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Container created 32m Normal Pulled pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 32m Warning Unhealthy pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Startup probe failed: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused 32m Warning ProbeError pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Startup probe error: Get "http://10.134.0.232:15021/healthz/ready": dial tcp 10.134.0.232:15021: connect: connection refused... 31m Normal Killing pod/e2e-inf-a-554759-openshift-default-69c6b6cbdf-gzhkq Stopping container istio-proxy 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-a39e7e-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-a39e7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 31m Warning Unhealthy pod/payload-processing-e2e-inf-a-554759-db5b8c5dc-mvxbd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-554759-5f8879f7c9-chpfd Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal Pulled pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal Started pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Container started 31m Normal Created pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Container created 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-599291-openshift-default Scaled up replica set e2e-shared-b-w1-599291-openshift-default-69f444c65c from 1 to 2 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-599291-openshift-default-69f444c65c Created pod: e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt 31m Normal AddedInterface pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Add eth0 [10.132.0.228/23] from ovn-kubernetes 31m Warning ProbeError pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Startup probe error: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused... 31m Warning Unhealthy pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Startup probe failed: Get "http://10.132.0.228:15021/healthz/ready": dial tcp 10.132.0.228:15021: connect: connection refused 31m Warning FailedToUpdateEndpoint endpoints/payload-processing-e2e-inf-b-35b584 Failed to update endpoint openshift-ingress/payload-processing-e2e-inf-b-35b584: Operation cannot be fulfilled on endpoints "payload-processing-e2e-inf-b-35b584": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-processing-e2e-inf-b-35b584, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 8da05d36-5ac5-4748-b25f-c396540aa68c, UID in object meta: 31m Normal Killing pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Stopping container payload-pre-processing 31m Normal Killing pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Stopping container payload-processing 31m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-shared-a-w4-a39e7e-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-shared-a-w4-a39e7e-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 31m Normal ScalingReplicaSet deployment/e2e-shared-a-w0-6a3533-openshift-default Scaled up replica set e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9 from 2 to 3 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w0-6a3533-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 31m Normal SuccessfulCreate replicaset/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9 Created pod: e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 31m Normal Created pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Container created 31m Normal Pulled pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Add eth0 [10.134.0.245/23] from ovn-kubernetes 31m Normal Started pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Container started 31m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-35b584-655c54f7ff-tqq44 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning ProbeError pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Startup probe error: Get "http://10.134.0.245:15021/healthz/ready": dial tcp 10.134.0.245:15021: connect: connection refused... 31m Warning Unhealthy pod/payload-processing-e2e-inf-b-35b584-5b757cf5d5-ntgf8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 31m Warning Unhealthy pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Startup probe failed: Get "http://10.134.0.245:15021/healthz/ready": dial tcp 10.134.0.245:15021: connect: connection refused 31m Normal Killing pod/e2e-inf-b-35b584-openshift-default-556b575787-8lxl6 Stopping container istio-proxy 31m Normal Killing pod/e2e-inf-b-35b584-openshift-default-556b575787-2zj26 Stopping container istio-proxy 31m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 31m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-14ba5a-openshift-default Scaled up replica set e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f from 1 to 2 31m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f Created pod: e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 31m Normal Pulled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 31m Normal AddedInterface pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Add eth0 [10.134.0.246/23] from ovn-kubernetes 31m Normal Created pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Container created 31m Normal Started pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Container started 31m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Startup probe failed: Get "http://10.134.0.246:15021/healthz/ready": dial tcp 10.134.0.246:15021: connect: connection refused 31m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Startup probe error: Get "http://10.134.0.246:15021/healthz/ready": dial tcp 10.134.0.246:15021: connect: connection refused... 30m Normal Killing pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Stopping container payload-pre-processing 30m Normal Killing pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Stopping container payload-processing 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Warning Unhealthy pod/payload-pre-processing-e2e-inf-a-1ddade-6b7b57b595-swkz7 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 30m Normal Killing pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-jndsx Stopping container istio-proxy 30m Normal Killing pod/e2e-inf-a-1ddade-openshift-default-5df978b75c-nx9hn Stopping container istio-proxy 30m Warning Unhealthy pod/payload-processing-e2e-inf-a-1ddade-7c4f444ff9-plwht Readiness probe failed: timeout: failed to connect service "10.132.0.173:50052" within 1s: context deadline exceeded 30m Normal Killing pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Stopping container payload-processing 30m Normal Killing pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Stopping container payload-pre-processing 29m Normal AddedInterface pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Add eth0 [10.133.0.173/23] from ovn-kubernetes 29m Normal Started pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Container started 29m Normal Created pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Container created 29m Normal Pulled pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 29m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f Created pod: e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j 29m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-abdcb1-openshift-default Scaled up replica set e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f from 1 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-abdcb1-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 29m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 5; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5d84bc6b95 from 6 to 5 29m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Deleted pod: maas-default-gateway-openshift-default-5d84bc6b95-dlwlz 29m Normal Killing pod/maas-default-gateway-openshift-default-5d84bc6b95-dlwlz Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Startup probe failed: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused 29m Warning ProbeError pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Startup probe error: Get "http://10.133.0.173:15021/healthz/ready": dial tcp 10.133.0.173:15021: connect: connection refused... 29m Normal ScalingReplicaSet deployment/e2e-worker-w6-aabeef-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w6-aabeef-openshift-default-6b8d7f885c from 3 to 2 29m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w6-aabeef-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c Deleted pod: e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 29m Warning Unhealthy pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal Killing pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Stopping container istio-proxy 29m Warning ProbeError pod/e2e-inf-b-72c210-openshift-default-5c9fc9cb8c-nf9hz Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/payload-processing-e2e-inf-b-72c210-68bd768b5-tt4vs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Normal ScalingReplicaSet deployment/e2e-ipp-a-20b8a4-openshift-default Scaled up replica set e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945 from 0 to 1 29m Normal SuccessfulCreate replicaset/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945 Created pod: e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj 29m Normal AddedLabel gateway/e2e-ipp-a-20b8a4 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-20b8a4 29m Normal AddedInterface pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Add eth0 [10.134.0.247/23] from ovn-kubernetes 29m Warning Unhealthy pod/payload-pre-processing-e2e-inf-b-72c210-5f66dbcb59-hmtc2 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 29m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Readiness probe failed: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused 29m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-gqd85 Readiness probe error: Get "http://10.132.0.71:15021/healthz/ready": dial tcp 10.132.0.71:15021: connect: connection refused... 29m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-20b8a4 Scaled up replica set payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555 from 0 to 1 29m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555 Error creating: pods "payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-20b8a4: serviceaccount "payload-processing-e2e-ipp-a-20b8a4" not found 29m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-20b8a4 Scaled up replica set payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f from 0 to 1 29m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555 Created pod: payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 29m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f Created pod: payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 29m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f Error creating: pods "payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-20b8a4: serviceaccount "payload-processing-e2e-ipp-a-20b8a4" not found 29m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 Add eth0 [10.132.0.234/23] from ovn-kubernetes 29m Normal Created pod/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 Container created 29m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal Created pod/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 Container created 29m Normal Started pod/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 Container started 29m Normal Started pod/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 Container started 29m Normal Pulled pod/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 29m Normal AddedInterface pod/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 Add eth0 [10.132.0.233/23] from ovn-kubernetes 29m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt Stopping container istio-proxy 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 3; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-g4gmt 29m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default Scaled down replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 4 to 3 29m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Stopping container istio-proxy 29m Normal SuccessfulDelete replicaset/e2e-worker-w5-6784d6-openshift-default-5789cc64d5 Deleted pod: e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns 29m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Stopping container istio-proxy 29m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-worker-w3-c82140-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w3-c82140-openshift-default-6c6b7c6899 from 3 to 2 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w3-c82140-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w0-7b3ddd-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w4-8b2f0a-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal ScalingReplicaSet deployment/e2e-worker-w0-7b3ddd-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf from 3 to 2 29m Normal SuccessfulDelete replicaset/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf Deleted pod: e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj 29m Normal ScalingReplicaSet deployment/e2e-worker-w4-8b2f0a-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w4-8b2f0a-openshift-default-64797b57bc from 3 to 2 29m Normal SuccessfulDelete replicaset/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc Deleted pod: e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w5-6784d6-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w3-c82140-openshift-default-6c6b7c6899 Deleted pod: e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 29m Normal ScalingReplicaSet deployment/e2e-worker-w5-6784d6-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w5-6784d6-openshift-default-5789cc64d5 from 3 to 2 29m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Stopping container istio-proxy 29m Normal ScalingReplicaSet deployment/e2e-worker-w2-5286ea-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w2-5286ea-openshift-default-646545bd4c from 3 to 2 29m Normal SuccessfulDelete replicaset/e2e-worker-w2-5286ea-openshift-default-646545bd4c Deleted pod: e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w1-d85d82-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Readiness probe failed: HTTP probe failed with statuscode: 503 29m Normal ScalingReplicaSet deployment/e2e-worker-w1-d85d82-openshift-default (combined from similar events): Scaled down replica set e2e-worker-w1-d85d82-openshift-default-7b77944d8f from 3 to 2 29m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-9hgns Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal SuccessfulRescale horizontalpodautoscaler/e2e-worker-w2-5286ea-openshift-default (combined from similar events): New size: 2; reason: All metrics below target 29m Normal SuccessfulDelete replicaset/e2e-worker-w1-d85d82-openshift-default-7b77944d8f Deleted pod: e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq 29m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Stopping container istio-proxy 29m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-ll7vq Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-w9476 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-6wcld Readiness probe error: HTTP probe failed with statuscode: 503... 29m Normal Killing pod/payload-processing-e2e-ipp-a-20b8a4-6dcb599f9f-hwmv5 Stopping container payload-processing 29m Normal Killing pod/payload-pre-processing-e2e-ipp-a-20b8a4-6b7c898555-kb8f2 Stopping container payload-pre-processing 29m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Readiness probe error: HTTP probe failed with statuscode: 503... 29m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-swsf8 Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Readiness probe failed: HTTP probe failed with statuscode: 503 29m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-4bmkj Readiness probe error: HTTP probe failed with statuscode: 503... 28m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-bb782 Stopping container discovery 28m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-bb782 28m Normal AddedInterface pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Add eth0 [10.132.0.239/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-4be675-openshift-default Scaled up replica set e2e-ipp-a-4be675-openshift-default-6cddb4cb85 from 0 to 1 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-4be675-openshift-default-6cddb4cb85 Created pod: e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 28m Normal AddedLabel gateway/e2e-ipp-a-4be675 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-4be675 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4be675 Scaled up replica set payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59 from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4be675 Scaled up replica set payload-processing-e2e-ipp-a-4be675-7d6588dfb7 from 0 to 1 28m Normal AddedLabel gateway/e2e-ipp-a-a81979 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-a81979 28m Normal ScalingReplicaSet deployment/e2e-ipp-a-a81979-openshift-default Scaled up replica set e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4be675-7d6588dfb7 Created pod: payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 28m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59 Created pod: payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59 Error creating: pods "payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be675: serviceaccount "payload-processing-e2e-ipp-a-4be675" not found 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4be675-7d6588dfb7 Error creating: pods "payload-processing-e2e-ipp-a-4be675-7d6588dfb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be675: serviceaccount "payload-processing-e2e-ipp-a-4be675" not found 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 Add eth0 [10.132.0.241/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt Add eth0 [10.132.0.242/23] from ovn-kubernetes 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt Container created 28m Normal Started pod/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 Container started 28m Normal Created pod/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt Container started 28m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Add eth0 [10.132.0.240/23] from ovn-kubernetes 28m Normal Pulled pod/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf Error creating: pods "payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a81979: serviceaccount "payload-processing-e2e-ipp-a-a81979" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf Created pod: payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a81979 Scaled up replica set payload-pre-processing-e2e-ipp-a-a81979-58d448b487 from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a81979 Scaled up replica set payload-processing-e2e-ipp-a-a81979-54f6f9b7cf from 0 to 1 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a81979-58d448b487 Created pod: payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a81979-58d448b487 Error creating: pods "payload-pre-processing-e2e-ipp-a-a81979-58d448b487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a81979: serviceaccount "payload-processing-e2e-ipp-a-a81979" not found 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Created pod/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f Container created 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f Add eth0 [10.132.0.244/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f Container started 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx Add eth0 [10.132.0.243/23] from ovn-kubernetes 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx Container created 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-20b8a4 Scaled up replica set payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667 from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88 Error creating: pods "payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-20b8a4: serviceaccount "payload-processing-e2e-ipp-a-20b8a4" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88 Created pod: payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-20b8a4 Scaled up replica set payload-processing-e2e-ipp-a-20b8a4-5d968c6d88 from 0 to 1 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667 Error creating: pods "payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-20b8a4: serviceaccount "payload-processing-e2e-ipp-a-20b8a4" not found 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667 Created pod: payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w 28m Normal Pulling pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal Pulling pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Add eth0 [10.132.0.245/23] from ovn-kubernetes 28m Normal AddedInterface pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Add eth0 [10.132.0.246/23] from ovn-kubernetes 28m Normal Started pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Container started 28m Normal Pulled pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.002s (2.002s including waiting). Image size: 166620835 bytes. 28m Normal Created pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 2.071s (2.071s including waiting). Image size: 166620835 bytes. 28m Normal Created pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Container created 28m Normal Started pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Container started 28m Normal ScalingReplicaSet deployment/e2e-ipp-b-8a47d6-openshift-default Scaled up replica set e2e-ipp-b-8a47d6-openshift-default-5b466b7b45 from 0 to 1 28m Normal SuccessfulCreate replicaset/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45 Created pod: e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn 28m Normal AddedLabel gateway/e2e-ipp-b-8a47d6 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-8a47d6 28m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4be675-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4be675-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 28m Normal AddedInterface pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Add eth0 [10.132.0.247/23] from ovn-kubernetes 28m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-8a47d6 Scaled up replica set payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5 from 0 to 1 28m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-8a47d6 Scaled up replica set payload-processing-e2e-ipp-b-8a47d6-78d764f4c from 0 to 1 28m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-8a47d6-78d764f4c Error creating: pods "payload-processing-e2e-ipp-b-8a47d6-78d764f4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8a47d6: serviceaccount "payload-processing-e2e-ipp-b-8a47d6" not found 28m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-8a47d6-78d764f4c Created pod: payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss 28m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5 Created pod: payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj 28m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5 Error creating: pods "payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8a47d6: serviceaccount "payload-processing-e2e-ipp-b-8a47d6" not found 28m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj Add eth0 [10.134.0.252/23] from ovn-kubernetes 28m Normal Started pod/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj Container started 28m Normal Started pod/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss Container started 28m Normal Created pod/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss Container created 28m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal Pulled pod/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 28m Normal AddedInterface pod/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss Add eth0 [10.134.0.253/23] from ovn-kubernetes 28m Normal Created pod/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj Container created 28m Normal Created pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Container created 28m Normal Pulled pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 28m Warning ProbeError pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Readiness probe error: Get "http://10.134.0.247:15021/healthz/ready": read tcp 10.134.0.2:45322->10.134.0.247:15021: read: connection reset by peer... 28m Warning Unhealthy pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Readiness probe failed: Get "http://10.134.0.247:15021/healthz/ready": read tcp 10.134.0.2:45322->10.134.0.247:15021: read: connection reset by peer 28m Normal Started pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Container started 28m Warning ProbeError pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Startup probe error: Get "http://10.134.0.247:15021/healthz/ready": dial tcp 10.134.0.247:15021: connect: connection refused... 28m Warning Unhealthy pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Startup probe failed: Get "http://10.134.0.247:15021/healthz/ready": dial tcp 10.134.0.247:15021: connect: connection refused 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4be675-74b5d6fd59-dt6lt Stopping container payload-pre-processing 28m Normal Killing pod/payload-processing-e2e-ipp-a-4be675-7d6588dfb7-c4dt4 Stopping container payload-processing 28m Normal Killing pod/payload-processing-e2e-ipp-a-a81979-54f6f9b7cf-gnk5f Stopping container payload-processing 28m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a81979-58d448b487-wvgnx Stopping container payload-pre-processing 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-4be675 Scaled up replica set payload-processing-e2e-ipp-a-4be675-85cb959c76 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-4be675-85cb959c76 Created pod: payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-4be675-85cb959c76 Error creating: pods "payload-processing-e2e-ipp-a-4be675-85cb959c76-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be675: serviceaccount "payload-processing-e2e-ipp-a-4be675" not found 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c Error creating: pods "payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-4be675: serviceaccount "payload-processing-e2e-ipp-a-4be675" not found 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c Created pod: payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-4be675 Scaled up replica set payload-pre-processing-e2e-ipp-a-4be675-846c64f56c from 0 to 1 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 Add eth0 [10.132.0.249/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 Add eth0 [10.132.0.250/23] from ovn-kubernetes 27m Normal Pulled pod/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 Container started 27m Normal Created pod/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 Container created 27m Normal Started pod/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 Container started 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 Container created 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495 Error creating: pods "payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a81979: serviceaccount "payload-processing-e2e-ipp-a-a81979" not found 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-a81979 Scaled up replica set payload-pre-processing-e2e-ipp-a-a81979-b88bfc495 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495 Created pod: payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-a81979-784f44796d Created pod: payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-a81979-784f44796d Error creating: pods "payload-processing-e2e-ipp-a-a81979-784f44796d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-a81979: serviceaccount "payload-processing-e2e-ipp-a-a81979" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-a81979 Scaled up replica set payload-processing-e2e-ipp-a-a81979-784f44796d from 0 to 1 27m Normal Started pod/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf Container started 27m Normal Pulled pod/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Started pod/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 Container started 27m Normal Created pod/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 Container created 27m Normal AddedInterface pod/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf Add eth0 [10.132.0.251/23] from ovn-kubernetes 27m Normal Created pod/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf Container created 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 Add eth0 [10.132.0.252/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Killing pod/payload-pre-processing-e2e-ipp-b-8a47d6-5464c779c5-wh6zj Stopping container payload-pre-processing 27m Normal Killing pod/payload-processing-e2e-ipp-b-8a47d6-78d764f4c-mqbss Stopping container payload-processing 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-20b8a4-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-20b8a4-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Normal AddedLabel gateway/e2e-ipp-b-dafbce Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-dafbce 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-dafbce-openshift-default Scaled up replica set e2e-ipp-b-dafbce-openshift-default-6fb8c547ff from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff Created pod: e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td 27m Normal AddedInterface pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Add eth0 [10.134.0.254/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655 Error creating: pods "payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-dafbce: serviceaccount "payload-processing-e2e-ipp-b-dafbce" not found 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-8a47d6-openshift-default Scaled up replica set e2e-ipp-b-8a47d6-openshift-default-5b466b7b45 from 1 to 4 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7 Created pod: payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 27m Normal Started pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Container started 27m Normal Created pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Container created 27m Normal Pulled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Add eth0 [10.133.0.175/23] from ovn-kubernetes 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-403dff-openshift-default-7656f86cf5 Created pod: e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f 27m Normal AddedInterface pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Add eth0 [10.134.0.255/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Container created 27m Normal Started pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Container started 27m Normal ScalingReplicaSet deployment/e2e-ipp-b-403dff-openshift-default Scaled up replica set e2e-ipp-b-403dff-openshift-default-7656f86cf5 from 0 to 1 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45 Created pod: e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45 Created pod: e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm 27m Normal SuccessfulCreate replicaset/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45 Created pod: e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-dafbce Scaled up replica set payload-pre-processing-e2e-ipp-b-dafbce-79999dc655 from 0 to 1 27m Normal AddedLabel gateway/e2e-ipp-b-403dff Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-403dff 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-dafbce Scaled up replica set payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7 from 0 to 1 27m Normal AddedInterface pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Add eth0 [10.134.1.0/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7 Error creating: pods "payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-dafbce: serviceaccount "payload-processing-e2e-ipp-b-dafbce" not found 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-b-8a47d6-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 27m Normal Created pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Container created 27m Normal SuccessfulCreate replicaset/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99 Created pod: e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z 27m Normal ScalingReplicaSet deployment/e2e-shared-b-w4-35d13e-openshift-default Scaled up replica set e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99 from 1 to 2 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn Add eth0 [10.134.1.1/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655 Created pod: payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w4-35d13e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal Started pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Container started 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 Add eth0 [10.132.0.255/23] from ovn-kubernetes 27m Normal AddedInterface pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Add eth0 [10.133.0.176/23] from ovn-kubernetes 27m Normal Created pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Container created 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn Container created 27m Normal Pulled pod/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Container started 27m Normal Pulled pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Add eth0 [10.134.1.2/23] from ovn-kubernetes 27m Normal Created pod/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 Container started 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn Container started 27m Normal ScalingReplicaSet deployment/e2e-ipp-a-a81979-openshift-default Scaled up replica set e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf from 1 to 4 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 27m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Readiness probe failed: Get "http://10.132.0.247:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l 27m Warning ProbeError pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Readiness probe error: Get "http://10.132.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Readiness probe error: Get "http://10.132.0.247:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 27m Warning Unhealthy pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Readiness probe failed: Get "http://10.132.0.239:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 27m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Container created 27m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Add eth0 [10.133.0.177/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Container created 27m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Container started 27m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Add eth0 [10.132.1.0/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Container started 27m Normal Pulled pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Add eth0 [10.134.1.3/23] from ovn-kubernetes 27m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Container created 27m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Container created 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7 Error creating: pods "payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-403dff: serviceaccount "payload-processing-e2e-ipp-b-403dff" not found 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-403dff-74c67b65b9 Error creating: pods "payload-processing-e2e-ipp-b-403dff-74c67b65b9-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-403dff: serviceaccount "payload-processing-e2e-ipp-b-403dff" not found 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-403dff Scaled up replica set payload-processing-e2e-ipp-b-403dff-74c67b65b9 from 0 to 1 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7 Created pod: payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z 27m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Container started 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-403dff-74c67b65b9 Created pod: payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch 27m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Container started 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-403dff Scaled up replica set payload-pre-processing-e2e-ipp-b-403dff-57764cfb7 from 0 to 1 27m Normal Started pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Container started 27m Normal Created pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Container created 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Add eth0 [10.132.1.4/23] from ovn-kubernetes 27m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-97927b-openshift-default Scaled up replica set e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c from 1 to 2 27m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-97927b-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 27m Normal Pulled pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c Created pod: e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc 27m Normal Created pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Container created 27m Normal Started pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Container started 27m Normal Pulled pod/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Startup probe error: Get "http://10.134.1.0:15021/healthz/ready": dial tcp 10.134.1.0:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Startup probe failed: Get "http://10.134.1.0:15021/healthz/ready": dial tcp 10.134.1.0:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Startup probe failed: Get "http://10.134.0.255:15021/healthz/ready": dial tcp 10.134.0.255:15021: connect: connection refused 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch Add eth0 [10.132.1.5/23] from ovn-kubernetes 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 27m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Startup probe error: Get "http://10.134.0.255:15021/healthz/ready": dial tcp 10.134.0.255:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Startup probe failed: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Startup probe error: Get "http://10.133.0.175:15021/healthz/ready": dial tcp 10.133.0.175:15021: connect: connection refused... 27m Normal AddedInterface pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Add eth0 [10.132.1.6/23] from ovn-kubernetes 27m Warning Unhealthy pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Startup probe failed: Get "http://10.134.1.2:15021/healthz/ready": dial tcp 10.134.1.2:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Startup probe error: Get "http://10.134.1.2:15021/healthz/ready": dial tcp 10.134.1.2:15021: connect: connection refused... 27m Normal Created pod/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch Container created 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Container created 27m Normal Pulled pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 27m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Startup probe error: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Startup probe error: Get "http://10.132.1.0:15021/healthz/ready": dial tcp 10.132.1.0:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Startup probe failed: Get "http://10.132.1.0:15021/healthz/ready": dial tcp 10.132.1.0:15021: connect: connection refused 27m Normal Started pod/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch Container started 27m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Startup probe failed: Get "http://10.133.0.177:15021/healthz/ready": dial tcp 10.133.0.177:15021: connect: connection refused 27m Normal Created pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Container started 27m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Startup probe failed: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Startup probe error: Get "http://10.132.0.247:15021/healthz/ready": dial tcp 10.132.0.247:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Startup probe error: Get "http://10.134.1.3:15021/healthz/ready": dial tcp 10.134.1.3:15021: connect: connection refused... 27m Warning Unhealthy pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Startup probe failed: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Startup probe failed: Get "http://10.134.1.3:15021/healthz/ready": dial tcp 10.134.1.3:15021: connect: connection refused 27m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Startup probe failed: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused 27m Warning ProbeError pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Startup probe error: Get "http://10.132.0.239:15021/healthz/ready": dial tcp 10.132.0.239:15021: connect: connection refused... 27m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Startup probe error: Get "http://10.132.0.240:15021/healthz/ready": dial tcp 10.132.0.240:15021: connect: connection refused... 27m Normal Started pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Container started 27m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-8a47d6 Scaled up replica set payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447 from 0 to 1 27m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-8a47d6 Scaled up replica set payload-processing-e2e-ipp-b-8a47d6-78df5798b8 from 0 to 1 27m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-8a47d6-78df5798b8 Error creating: pods "payload-processing-e2e-ipp-b-8a47d6-78df5798b8-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8a47d6: serviceaccount "payload-processing-e2e-ipp-b-8a47d6" not found 27m Normal Pulled pod/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj Add eth0 [10.132.1.8/23] from ovn-kubernetes 27m Normal AddedInterface pod/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv Add eth0 [10.132.1.7/23] from ovn-kubernetes 27m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447 Error creating: pods "payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-8a47d6: serviceaccount "payload-processing-e2e-ipp-b-8a47d6" not found 27m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-8a47d6-78df5798b8 Created pod: payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv 27m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447 Created pod: payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj 27m Normal Created pod/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv Container created 27m Normal Started pod/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv Container started 27m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 27m Normal Created pod/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj Container created 27m Normal Started pod/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj Container started 27m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 27m Warning Unhealthy pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Startup probe failed: Get "http://10.132.1.6:15021/healthz/ready": dial tcp 10.132.1.6:15021: connect: connection refused 27m Warning ProbeError pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Startup probe error: Get "http://10.132.1.6:15021/healthz/ready": dial tcp 10.132.1.6:15021: connect: connection refused... 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-599291-openshift-default Scaled up replica set e2e-shared-b-w1-599291-openshift-default-69f444c65c from 2 to 3 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w1-599291-openshift-default-69f444c65c Created pod: e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 26m Normal Created pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Container created 26m Normal Started pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Container started 26m Normal Pulled pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Add eth0 [10.133.0.179/23] from ovn-kubernetes 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-8a47d6-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-8a47d6-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 26m Warning Unhealthy pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Startup probe failed: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused 26m Warning ProbeError pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Startup probe error: Get "http://10.133.0.179:15021/healthz/ready": dial tcp 10.133.0.179:15021: connect: connection refused... 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-4be675-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-4be675-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Normal Killing pod/payload-pre-processing-e2e-ipp-a-20b8a4-575cbc6667-4wx5w Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-a-20b8a4-5d968c6d88-6rkqk Stopping container payload-processing 26m Normal SuccessfulDelete replicaset/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc Deleted pod: e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w6-15dc2f-openshift-default (combined from similar events): Scaled down replica set e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc from 3 to 2 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w6-15dc2f-openshift-default New size: 2; reason: All metrics below target 26m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-7lxzf Stopping container istio-proxy 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default New size: 8; reason: All metrics below target 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-a81979-openshift-default Scaled up replica set e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf from 4 to 8 26m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Add eth0 [10.133.0.181/23] from ovn-kubernetes 26m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Container created 26m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Container started 26m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Add eth0 [10.134.1.4/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Container started 26m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Container created 26m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Container created 26m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Add eth0 [10.134.1.5/23] from ovn-kubernetes 26m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Container started 26m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Container created 26m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Container started 26m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Add eth0 [10.132.1.13/23] from ovn-kubernetes 26m Normal Killing pod/e2e-ipp-a-20b8a4-openshift-default-5cd4d5f945-lt9hj Stopping container istio-proxy 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-dafbce-79999dc655-cxnbn Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-dafbce-7f4cf56cc7-dgnx9 Stopping container payload-processing 26m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Startup probe error: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Startup probe failed: Get "http://10.132.1.13:15021/healthz/ready": dial tcp 10.132.1.13:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Startup probe error: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Startup probe failed: Get "http://10.133.0.181:15021/healthz/ready": dial tcp 10.133.0.181:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Startup probe error: Get "http://10.134.1.5:15021/healthz/ready": dial tcp 10.134.1.5:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Startup probe failed: Get "http://10.134.1.5:15021/healthz/ready": dial tcp 10.134.1.5:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Startup probe error: Get "http://10.134.1.4:15021/healthz/ready": dial tcp 10.134.1.4:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Startup probe failed: Get "http://10.134.1.4:15021/healthz/ready": dial tcp 10.134.1.4:15021: connect: connection refused 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-403dff-74c67b65b9-smzch Stopping container payload-processing 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf Created pod: e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-a81979-openshift-default Scaled up replica set e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf from 8 to 9 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-ipp-a-a81979-openshift-default New size: 9; reason: All metrics below target 26m Normal AddedInterface pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Add eth0 [10.133.0.182/23] from ovn-kubernetes 26m Normal Created pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Container created 26m Normal Pulled pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Container started 26m Warning ProbeError pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Readiness probe error: Get "http://10.133.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 26m Warning Unhealthy pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Readiness probe failed: Get "http://10.133.0.176:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 26m Warning Unhealthy pod/payload-pre-processing-e2e-ipp-b-403dff-57764cfb7-vld2z Readiness probe failed: timeout: failed to connect service "10.132.1.4:50052" within 1s: context deadline exceeded 26m Normal Created pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Container created 26m Normal Started pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Container started 26m Normal Created pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Container created 26m Normal Pulled pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Pulled pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Started pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Container started 26m Normal Killing pod/payload-pre-processing-e2e-ipp-b-8a47d6-7556fc4447-wtjfj Stopping container payload-pre-processing 26m Normal Killing pod/payload-processing-e2e-ipp-b-8a47d6-78df5798b8-dxtmv Stopping container payload-processing 26m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Startup probe error: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Startup probe failed: Get "http://10.133.0.182:15021/healthz/ready": dial tcp 10.133.0.182:15021: connect: connection refused 26m Normal AddedLabel gateway/e2e-ipp-a-e50b22 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-e50b22 26m Normal ScalingReplicaSet deployment/e2e-ipp-a-e50b22-openshift-default Scaled up replica set e2e-ipp-a-e50b22-openshift-default-ccc6dd79f from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f Created pod: e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz 26m Normal AddedInterface pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Add eth0 [10.132.1.17/23] from ovn-kubernetes 26m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-403dff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-403dff-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 26m Warning Unhealthy pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Startup probe failed: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused 26m Warning ProbeError pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Startup probe error: Get "http://10.133.0.176:15021/healthz/ready": dial tcp 10.133.0.176:15021: connect: connection refused... 26m Warning ProbeError pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Startup probe error: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused... 26m Warning Unhealthy pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Startup probe failed: Get "http://10.134.0.254:15021/healthz/ready": dial tcp 10.134.0.254:15021: connect: connection refused 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e50b22 Scaled up replica set payload-pre-processing-e2e-ipp-a-e50b22-6978849c67 from 0 to 1 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67 Created pod: payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e50b22-5f4c877789 Created pod: payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e50b22-5f4c877789 Error creating: pods "payload-processing-e2e-ipp-a-e50b22-5f4c877789-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50b22: serviceaccount "payload-processing-e2e-ipp-a-e50b22" not found 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e50b22 Scaled up replica set payload-processing-e2e-ipp-a-e50b22-5f4c877789 from 0 to 1 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67 Error creating: pods "payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50b22: serviceaccount "payload-processing-e2e-ipp-a-e50b22" not found 26m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 Add eth0 [10.132.1.18/23] from ovn-kubernetes 26m Normal Started pod/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 Container started 26m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f Created pod: e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f 26m Normal Killing pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Stopping container istio-proxy 26m Normal Created pod/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx Container created 26m Normal Started pod/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx Container started 26m Normal Pulled pod/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx Add eth0 [10.134.1.6/23] from ovn-kubernetes 26m Normal Created pod/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 Container created 26m Normal Killing pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-qgqcn Stopping container istio-proxy 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal Killing pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Stopping container istio-proxy 26m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 26m Normal Killing pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Stopping container istio-proxy 26m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-14ba5a-openshift-default Scaled up replica set e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f from 2 to 3 26m Normal AddedInterface pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Add eth0 [10.133.0.183/23] from ovn-kubernetes 26m Normal Pulled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal Created pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Container created 26m Normal Started pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Container started 26m Normal Pulled pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 26m Normal AddedInterface pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Add eth0 [10.133.0.184/23] from ovn-kubernetes 26m Normal ScalingReplicaSet deployment/e2e-shared-a-w1-97927b-openshift-default Scaled up replica set e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c from 2 to 3 26m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w1-97927b-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 26m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-dafbce Scaled up replica set payload-processing-e2e-ipp-b-dafbce-f6dc47458 from 0 to 1 26m Normal SuccessfulCreate replicaset/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c Created pod: e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv 26m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-dafbce Scaled up replica set payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5 from 0 to 1 26m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-dafbce-f6dc47458 Error creating: pods "payload-processing-e2e-ipp-b-dafbce-f6dc47458-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-dafbce: serviceaccount "payload-processing-e2e-ipp-b-dafbce" not found 26m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-dafbce-f6dc47458 Created pod: payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw 26m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5 Error creating: pods "payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-dafbce: serviceaccount "payload-processing-e2e-ipp-b-dafbce" not found 26m Normal Started pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Container started 26m Normal Created pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Container created 26m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5 Created pod: payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz Container created 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz Add eth0 [10.132.1.20/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-w6d46 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw Add eth0 [10.132.1.21/23] from ovn-kubernetes 25m Normal Created pod/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw Container created 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz Container started 25m Normal Started pod/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw Container started 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Normal Pulled pod/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 25m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-v6bjm Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Readiness probe failed: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 25m Warning ProbeError pod/e2e-ipp-b-8a47d6-openshift-default-5b466b7b45-7wkjh Readiness probe error: Get "http://10.134.1.0:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-dafbce-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-dafbce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 25m Normal SuccessfulCreate replicaset/istiod-openshift-gateway-7ddd9cfdc8 Created pod: istiod-openshift-gateway-7ddd9cfdc8-gk567 25m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled up replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 3 25m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 3; reason: cpu resource utilization (percentage of request) above target 25m Normal Created pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Container created 25m Normal Pulled pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Container image "registry.redhat.io/openshift-service-mesh/istio-pilot-rhel9@sha256:d5adf3101d4e14c710cd05afb3fa53498e3c5fb31a120f1bb007fba2e3b376f8" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Add eth0 [10.132.1.22/23] from ovn-kubernetes 25m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Startup probe error: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused... 25m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Startup probe failed: Get "http://10.133.0.183:15021/healthz/ready": dial tcp 10.133.0.183:15021: connect: connection refused 25m Normal Started pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Container started 25m Warning ProbeError pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Readiness probe error: HTTP probe failed with statuscode: 503... 25m Warning Unhealthy pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Readiness probe failed: HTTP probe failed with statuscode: 503 25m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-403dff-55687f99 Error creating: pods "payload-pre-processing-e2e-ipp-b-403dff-55687f99-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-403dff: serviceaccount "payload-processing-e2e-ipp-b-403dff" not found 25m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-403dff-55687f99 Created pod: payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg 25m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-403dff Scaled up replica set payload-pre-processing-e2e-ipp-b-403dff-55687f99 from 0 to 1 25m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-403dff-7f4fb76d66 Error creating: pods "payload-processing-e2e-ipp-b-403dff-7f4fb76d66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-403dff: serviceaccount "payload-processing-e2e-ipp-b-403dff" not found 25m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-403dff-7f4fb76d66 Created pod: payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b 25m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-403dff Scaled up replica set payload-processing-e2e-ipp-b-403dff-7f4fb76d66 from 0 to 1 25m Normal Killing pod/payload-processing-e2e-shared-a-w2-4f340c-74b9cf49cc-56k6d Stopping container payload-processing 25m Warning ProbeError pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Startup probe error: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused... 25m Normal Killing pod/payload-pre-processing-e2e-shared-a-w2-4f340c-5b7cf9b475-9ghl4 Stopping container payload-pre-processing 25m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Add eth0 [10.134.1.7/23] from ovn-kubernetes 25m Warning Unhealthy pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Startup probe failed: Get "http://10.133.0.184:15021/healthz/ready": dial tcp 10.133.0.184:15021: connect: connection refused 25m Normal AddedInterface pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Add eth0 [10.134.1.8/23] from ovn-kubernetes 25m Normal Pulling pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Normal Pulling pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Pulling image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-403dff-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-403dff-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Started pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Container started 25m Normal Started pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Container started 25m Normal Created pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Container created 25m Normal Pulled pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.456s (1.456s including waiting). Image size: 166620835 bytes. 25m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Successfully pulled image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" in 1.57s (1.57s including waiting). Image size: 166620835 bytes. 25m Normal Created pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Container created 25m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-dafbce-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-dafbce-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 25m Normal Killing pod/e2e-shared-a-w2-4f340c-openshift-default-7bb446c46c-scg7z Stopping container istio-proxy 25m Warning FailedToUpdateEndpoint endpoints/payload-pre-processing-e2e-ipp-a-4be675 Failed to update endpoint openshift-ingress/payload-pre-processing-e2e-ipp-a-4be675: Operation cannot be fulfilled on endpoints "payload-pre-processing-e2e-ipp-a-4be675": StorageError: invalid object, Code: 4, Key: /kubernetes.io/services/endpoints/openshift-ingress/payload-pre-processing-e2e-ipp-a-4be675, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 6825f150-fe91-4bcb-a9a2-abd4c73d1b25, UID in object meta: 25m Normal Killing pod/payload-pre-processing-e2e-shared-b-w2-b06b08-8688ccc55f-6r5d4 Stopping container payload-pre-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-4be675-846c64f56c-fwnb2 Stopping container payload-pre-processing 25m Normal Killing pod/payload-processing-e2e-shared-b-w2-b06b08-6cb9f4844c-9mkm6 Stopping container payload-processing 25m Normal Killing pod/payload-processing-e2e-ipp-a-4be675-85cb959c76-r8sz2 Stopping container payload-processing 25m Normal Killing pod/e2e-ipp-a-4be675-openshift-default-6cddb4cb85-jwrd8 Stopping container istio-proxy 25m Normal Killing pod/payload-processing-e2e-ipp-a-e50b22-5f4c877789-dwvbx Stopping container payload-processing 25m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e50b22-6978849c67-ppk29 Stopping container payload-pre-processing 25m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w5-14ba5a-openshift-default New size: 4; reason: cpu resource utilization (percentage of request) above target 25m Normal ScalingReplicaSet deployment/e2e-shared-b-w5-14ba5a-openshift-default Scaled up replica set e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f from 3 to 4 25m Normal Started pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Container started 25m Normal SuccessfulCreate replicaset/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f Created pod: e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx 25m Normal Created pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Container created 25m Normal Pulled pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 25m Normal AddedInterface pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Add eth0 [10.132.1.27/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/e2e-shared-b-w3-abdcb1-openshift-default Scaled up replica set e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f from 2 to 3 24m Normal SuccessfulCreate replicaset/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f Created pod: e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w3-abdcb1-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 24m Normal Created pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Container created 24m Normal Pulled pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Started pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Container started 24m Normal AddedInterface pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Add eth0 [10.132.1.29/23] from ovn-kubernetes 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-a39e7e-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c Created pod: e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk 24m Normal AddedInterface pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Add eth0 [10.132.1.30/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-a39e7e-openshift-default Scaled up replica set e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c from 1 to 2 24m Normal Created pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Container created 24m Normal Started pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Container started 24m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Startup probe failed: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused 24m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Startup probe error: Get "http://10.132.1.27:15021/healthz/ready": dial tcp 10.132.1.27:15021: connect: connection refused... 24m Normal Killing pod/e2e-shared-b-w2-b06b08-openshift-default-5cd658c6b5-srzrw Stopping container istio-proxy 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-e50b22 Scaled up replica set payload-processing-e2e-ipp-a-e50b22-7787cd6b66 from 0 to 1 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487 Created pod: payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 24m Warning ProbeError pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Startup probe error: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused... 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487 Error creating: pods "payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50b22: serviceaccount "payload-processing-e2e-ipp-a-e50b22" not found 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-e50b22 Scaled up replica set payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487 from 0 to 1 24m Warning Unhealthy pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Startup probe failed: Get "http://10.132.1.29:15021/healthz/ready": dial tcp 10.132.1.29:15021: connect: connection refused 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-e50b22-7787cd6b66 Error creating: pods "payload-processing-e2e-ipp-a-e50b22-7787cd6b66-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-e50b22: serviceaccount "payload-processing-e2e-ipp-a-e50b22" not found 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-e50b22-7787cd6b66 Created pod: payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz 24m Warning ProbeError pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Startup probe error: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Startup probe failed: Get "http://10.132.1.30:15021/healthz/ready": dial tcp 10.132.1.30:15021: connect: connection refused 24m Normal Started pod/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz Container started 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz Add eth0 [10.132.1.33/23] from ovn-kubernetes 24m Normal Pulled pod/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 Add eth0 [10.132.1.32/23] from ovn-kubernetes 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 Container created 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 Container started 24m Normal Created pod/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz Container created 24m Normal ScalingReplicaSet deployment/e2e-shared-a-w3-33c10a-openshift-default Scaled up replica set e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc from 1 to 2 24m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w3-33c10a-openshift-default New size: 2; reason: cpu resource utilization (percentage of request) above target 24m Normal SuccessfulCreate replicaset/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc Created pod: e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r 24m Normal Created pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Container created 24m Normal Started pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Container started 24m Normal AddedInterface pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Add eth0 [10.132.1.34/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Normal Killing pod/payload-processing-e2e-ipp-b-dafbce-f6dc47458-nbmcw Stopping container payload-processing 24m Normal Killing pod/payload-pre-processing-e2e-ipp-b-dafbce-754f9454f5-n7gdz Stopping container payload-pre-processing 24m Normal Killing pod/payload-pre-processing-e2e-worker-w2-5286ea-7fc5954f67-d8ws5 Stopping container payload-pre-processing 24m Normal Killing pod/payload-processing-e2e-worker-w2-5286ea-5f5f55bd9c-hmkdc Stopping container payload-processing 24m Normal AddedLabel gateway/e2e-ipp-a-9f9103 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-9f9103 24m Normal ScalingReplicaSet deployment/e2e-ipp-a-9f9103-openshift-default Scaled up replica set e2e-ipp-a-9f9103-openshift-default-cb9db67b7 from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-ipp-a-9f9103-openshift-default-cb9db67b7 Created pod: e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c 24m Warning ProbeError pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Startup probe error: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused... 24m Warning Unhealthy pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Startup probe failed: Get "http://10.132.1.34:15021/healthz/ready": dial tcp 10.132.1.34:15021: connect: connection refused 24m Normal AddedInterface pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Add eth0 [10.132.1.35/23] from ovn-kubernetes 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-9f9103 Scaled up replica set payload-processing-e2e-ipp-a-9f9103-5bd959fd85 from 0 to 1 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-9f9103 Scaled up replica set payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f from 0 to 1 24m Normal Created pod/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Pulled pod/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f Error creating: pods "payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9f9103: serviceaccount "payload-processing-e2e-ipp-a-9f9103" not found 24m Normal Started pod/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 Container started 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-9f9103-5bd959fd85 Created pod: payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 24m Normal Started pod/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 Container started 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-9f9103-5bd959fd85 Error creating: pods "payload-processing-e2e-ipp-a-9f9103-5bd959fd85-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9f9103: serviceaccount "payload-processing-e2e-ipp-a-9f9103" not found 24m Normal AddedInterface pod/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 Add eth0 [10.134.1.13/23] from ovn-kubernetes 24m Normal Created pod/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 Container created 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f Created pod: payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 Add eth0 [10.134.1.12/23] from ovn-kubernetes 24m Normal Killing pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Stopping container istio-proxy 24m Warning ProbeError pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-ipp-b-dafbce-openshift-default-6fb8c547ff-2d7td Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal AddedLabel gateway/e2e-ipp-b-5af6eb Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-5af6eb 24m Normal ScalingReplicaSet deployment/e2e-ipp-b-5af6eb-openshift-default Scaled up replica set e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5 from 0 to 1 24m Normal SuccessfulCreate replicaset/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5 Created pod: e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj 24m Normal AddedInterface pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Add eth0 [10.134.1.14/23] from ovn-kubernetes 24m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-5af6eb-698d589597 Error creating: pods "payload-processing-e2e-ipp-b-5af6eb-698d589597-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5af6eb: serviceaccount "payload-processing-e2e-ipp-b-5af6eb" not found 24m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-5af6eb Scaled up replica set payload-processing-e2e-ipp-b-5af6eb-698d589597 from 0 to 1 24m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f Error creating: pods "payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5af6eb: serviceaccount "payload-processing-e2e-ipp-b-5af6eb" not found 24m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f Created pod: payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 24m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-5af6eb-698d589597 Created pod: payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws 24m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-5af6eb Scaled up replica set payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f from 0 to 1 24m Normal Started pod/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws Container started 24m Normal Created pod/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws Container created 24m Normal Started pod/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 Container started 24m Normal Killing pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Stopping container payload-pre-processing 24m Normal Pulled pod/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal Killing pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Stopping container payload-processing 24m Normal AddedInterface pod/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws Add eth0 [10.133.0.185/23] from ovn-kubernetes 24m Normal Created pod/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 Container created 24m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 24m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 Add eth0 [10.134.1.15/23] from ovn-kubernetes 24m Normal Pulled pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 24m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e50b22-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e50b22-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 24m Normal Created pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Container created 24m Normal Started pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Container started 24m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Stopping container istio-proxy 24m Normal Killing pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-jq9jr Stopping container istio-proxy 24m Warning ProbeError pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Readiness probe error: HTTP probe failed with statuscode: 503... 24m Warning Unhealthy pod/e2e-worker-w2-5286ea-openshift-default-646545bd4c-hkngm Readiness probe failed: HTTP probe failed with statuscode: 503 24m Warning Unhealthy pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Startup probe failed: Get "http://10.132.1.17:15021/healthz/ready": dial tcp 10.132.1.17:15021: connect: connection refused 24m Warning ProbeError pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Startup probe error: Get "http://10.132.1.17:15021/healthz/ready": dial tcp 10.132.1.17:15021: connect: connection refused... 24m Warning Unhealthy pod/payload-processing-e2e-shared-a-w0-6a3533-68fb85b995-ljs96 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w0-6a3533-7fd5f5c8bc-2kwqj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 24m Warning Unhealthy pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Readiness probe failed: HTTP probe failed with statuscode: 503 24m Normal Killing pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Stopping container istio-proxy 24m Normal Killing pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-xb9m4 Stopping container istio-proxy 24m Warning ProbeError pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-d86kx Readiness probe error: HTTP probe failed with statuscode: 503... 24m Normal Killing pod/e2e-shared-a-w0-6a3533-openshift-default-74565f4bb9-66r8g Stopping container istio-proxy 23m Normal Killing pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-a-9f9103-85549bb9f-hk689 Stopping container payload-pre-processing 23m Normal Killing pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-a-9f9103-5bd959fd85-t6xg4 Stopping container payload-processing 23m Normal Killing pod/payload-pre-processing-e2e-ipp-b-5af6eb-65b55fd7f-mdvb2 Stopping container payload-pre-processing 23m Normal Killing pod/payload-processing-e2e-ipp-b-5af6eb-698d589597-km9ws Stopping container payload-processing 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-e50b22-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-e50b22-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6 Error creating: pods "payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9f9103: serviceaccount "payload-processing-e2e-ipp-a-9f9103" not found 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6 Created pod: payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-9f9103 Scaled up replica set payload-processing-e2e-ipp-a-9f9103-7b88cf99d6 from 0 to 1 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-9f9103 Scaled up replica set payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf Created pod: payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf Error creating: pods "payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-9f9103: serviceaccount "payload-processing-e2e-ipp-a-9f9103" not found 23m Normal Started pod/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d Container started 23m Normal Created pod/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d Container created 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d Add eth0 [10.134.1.17/23] from ovn-kubernetes 23m Normal Started pod/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 Container started 23m Normal Created pod/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 Add eth0 [10.134.1.18/23] from ovn-kubernetes 23m Normal Killing pod/e2e-shared-b-w0-612b46-openshift-default-79fc6676c5-b7mfb Stopping container istio-proxy 23m Warning Unhealthy pod/payload-processing-e2e-shared-b-w0-612b46-85d6fdfb9-vlqkr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning ProbeError pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Readiness probe error: Get "http://10.132.1.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 23m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w0-612b46-77485d8f9f-25996 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 23m Warning Unhealthy pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Readiness probe failed: Get "http://10.132.1.35:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-5af6eb Scaled up replica set payload-processing-e2e-ipp-b-5af6eb-8459557758 from 0 to 1 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-5af6eb-8459557758 Error creating: pods "payload-processing-e2e-ipp-b-5af6eb-8459557758-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5af6eb: serviceaccount "payload-processing-e2e-ipp-b-5af6eb" not found 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt Add eth0 [10.134.1.19/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-5af6eb-8459557758 Created pod: payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-5af6eb Scaled up replica set payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667 from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667 Created pod: payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667 Error creating: pods "payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-5af6eb: serviceaccount "payload-processing-e2e-ipp-b-5af6eb" not found 23m Normal Created pod/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz Container created 23m Normal Pulled pod/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt Container started 23m Normal Started pod/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz Container started 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz Add eth0 [10.134.1.20/23] from ovn-kubernetes 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt Container created 23m Normal Created pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Container created 23m Normal Pulled pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Started pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Container started 23m Normal Killing pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Stopping container payload-pre-processing 23m Normal Created pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Container created 23m Normal Started pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Container started 23m Normal Pulled pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 23m Normal Killing pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Stopping container payload-processing 23m Normal AddedLabel gateway/e2e-ipp-b-f9ca72 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-f9ca72 23m Normal AddedInterface pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Add eth0 [10.134.1.21/23] from ovn-kubernetes 23m Warning ProbeError pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Startup probe error: Get "http://10.132.1.35:15021/healthz/ready": dial tcp 10.132.1.35:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Startup probe failed: Get "http://10.132.1.35:15021/healthz/ready": dial tcp 10.132.1.35:15021: connect: connection refused 23m Normal ScalingReplicaSet deployment/e2e-ipp-b-f9ca72-openshift-default Scaled up replica set e2e-ipp-b-f9ca72-openshift-default-5c499594fc from 0 to 1 23m Normal SuccessfulCreate replicaset/e2e-ipp-b-f9ca72-openshift-default-5c499594fc Created pod: e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt 23m Warning ProbeError pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Startup probe error: Get "http://10.134.1.14:15021/healthz/ready": dial tcp 10.134.1.14:15021: connect: connection refused... 23m Warning Unhealthy pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Startup probe failed: Get "http://10.134.1.14:15021/healthz/ready": dial tcp 10.134.1.14:15021: connect: connection refused 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-9f9103-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-9f9103-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 23m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f Error creating: pods "payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9ca72: serviceaccount "payload-processing-e2e-ipp-b-f9ca72" not found 23m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f9ca72 Scaled up replica set payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5 from 0 to 1 23m Normal Pulled pod/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 Add eth0 [10.134.1.22/23] from ovn-kubernetes 23m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f Created pod: payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv 23m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5 Error creating: pods "payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9ca72: serviceaccount "payload-processing-e2e-ipp-b-f9ca72" not found 23m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f9ca72 Scaled up replica set payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f from 0 to 1 23m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5 Created pod: payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 23m Normal Started pod/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv Container started 23m Normal Created pod/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 Container created 23m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv Add eth0 [10.133.0.187/23] from ovn-kubernetes 23m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 23m Normal Created pod/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv Container created 23m Normal Started pod/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 Container started 23m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-5af6eb-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 23m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-5af6eb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w0-7b3ddd-75f47f5cc5-2c4hv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning Unhealthy pod/payload-processing-e2e-worker-w0-7b3ddd-5d6544b5f5-v6tzq Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-5af6eb-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-5af6eb-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 22m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-9f9103-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-9f9103-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-e50b22-6dc57f5487-hfgl6 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-e50b22-7787cd6b66-xfgwz Stopping container payload-processing 22m Normal Killing pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Stopping container istio-proxy 22m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Stopping container istio-proxy 22m Warning Unhealthy pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Killing pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Stopping container istio-proxy 22m Warning ProbeError pod/e2e-ipp-a-e50b22-openshift-default-ccc6dd79f-wz9lz Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Readiness probe failed: HTTP probe failed with statuscode: 503 22m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-m4hhj Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning ProbeError pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Readiness probe error: HTTP probe failed with statuscode: 503... 22m Warning Unhealthy pod/e2e-worker-w0-7b3ddd-openshift-default-5844f58fbf-7wvkp Readiness probe failed: HTTP probe failed with statuscode: 503 22m Normal Killing pod/payload-processing-e2e-ipp-b-f9ca72-f5cbf94c5-hd9n6 Stopping container payload-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f9ca72-5b6b49c45f-l9nnv Stopping container payload-pre-processing 22m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b Error creating: pods "payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9ca72: serviceaccount "payload-processing-e2e-ipp-b-f9ca72" not found 22m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc Add eth0 [10.134.1.30/23] from ovn-kubernetes 22m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f Error creating: pods "payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-f9ca72: serviceaccount "payload-processing-e2e-ipp-b-f9ca72" not found 22m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-f9ca72 Scaled up replica set payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b from 0 to 1 22m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b Created pod: payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc 22m Normal Pulled pod/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal AddedInterface pod/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc Add eth0 [10.132.1.57/23] from ovn-kubernetes 22m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f Created pod: payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc 22m Normal Started pod/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc Container started 22m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-f9ca72 Scaled up replica set payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f from 0 to 1 22m Normal Created pod/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc Container created 22m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 22m Normal Created pod/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc Container created 22m Normal Started pod/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc Container started 22m Normal Killing pod/payload-pre-processing-e2e-ipp-b-5af6eb-76b9475667-xphdt Stopping container payload-pre-processing 22m Normal Killing pod/payload-pre-processing-e2e-ipp-a-a81979-b88bfc495-zc2c9 Stopping container payload-pre-processing 22m Normal Killing pod/payload-processing-e2e-ipp-a-a81979-784f44796d-q4cdf Stopping container payload-processing 22m Normal Killing pod/payload-processing-e2e-ipp-b-5af6eb-8459557758-bl7fz Stopping container payload-processing 22m Normal Pulled pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 22m Normal Created pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Container created 22m Normal Started pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Container started 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-a-w4-a39e7e-openshift-default New size: 3; reason: cpu resource utilization (percentage of request) above target 21m Normal ScalingReplicaSet deployment/e2e-shared-a-w4-a39e7e-openshift-default Scaled up replica set e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c from 2 to 3 21m Normal SuccessfulCreate replicaset/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c Created pod: e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 21m Normal Killing pod/e2e-ipp-b-5af6eb-openshift-default-b4c5d69c5-8gxvj Stopping container istio-proxy 21m Normal Pulled pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 21m Normal Created pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Container created 21m Normal AddedInterface pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Add eth0 [10.133.0.197/23] from ovn-kubernetes 21m Normal Started pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Container started 21m Warning Unhealthy pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Startup probe failed: Get "http://10.134.1.21:15021/healthz/ready": dial tcp 10.134.1.21:15021: connect: connection refused 21m Warning ProbeError pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Startup probe error: Get "http://10.134.1.21:15021/healthz/ready": dial tcp 10.134.1.21:15021: connect: connection refused... 21m Warning ProbeError pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Startup probe error: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused... 21m Warning Unhealthy pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Startup probe failed: Get "http://10.133.0.197:15021/healthz/ready": dial tcp 10.133.0.197:15021: connect: connection refused 21m Normal Killing pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Stopping container payload-pre-processing 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f9ca72-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f9ca72-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 21m Normal Killing pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/e2e-shared-b-w1-599291-openshift-default Scaled down replica set e2e-shared-b-w1-599291-openshift-default-69f444c65c from 3 to 2 21m Normal SuccessfulRescale horizontalpodautoscaler/e2e-shared-b-w1-599291-openshift-default New size: 2; reason: All metrics below target 21m Normal SuccessfulDelete replicaset/e2e-shared-b-w1-599291-openshift-default-69f444c65c Deleted pod: e2e-shared-b-w1-599291-openshift-default-69f444c65c-k9v47 21m Normal Killing pod/payload-processing-e2e-ipp-a-9f9103-7b88cf99d6-t4t96 Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-ipp-a-9f9103-6bf8f4c9cf-vhf9d Stopping container payload-pre-processing 21m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-f9ca72-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-f9ca72-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 21m Normal Killing pod/e2e-ipp-a-9f9103-openshift-default-cb9db67b7-9s57c Stopping container istio-proxy 21m Normal ScalingReplicaSet deployment/e2e-ipp-a-586469-openshift-default Scaled up replica set e2e-ipp-a-586469-openshift-default-6cb999f45f from 0 to 1 21m Normal SuccessfulCreate replicaset/e2e-ipp-a-586469-openshift-default-6cb999f45f Created pod: e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m 21m Normal AddedLabel gateway/e2e-ipp-a-586469 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-a-586469 21m Normal AddedInterface pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Add eth0 [10.134.1.34/23] from ovn-kubernetes 21m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w1-97927b-84644994c9-8krrj Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Normal Killing pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-d2kmc Stopping container istio-proxy 21m Normal Killing pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-qbxs5 Stopping container istio-proxy 21m Warning Unhealthy pod/payload-processing-e2e-shared-a-w1-97927b-5f6bc8df95-m5x5m Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 21m Warning Unhealthy pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-shared-a-w1-97927b-openshift-default-5f4c5b677c-27vnv Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-586469 Scaled up replica set payload-processing-e2e-ipp-a-586469-7b8b9f445 from 0 to 1 21m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c Error creating: pods "payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-586469: serviceaccount "payload-processing-e2e-ipp-a-586469" not found 21m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-586469-7b8b9f445 Error creating: pods "payload-processing-e2e-ipp-a-586469-7b8b9f445-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-586469: serviceaccount "payload-processing-e2e-ipp-a-586469" not found 21m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-586469 Scaled up replica set payload-pre-processing-e2e-ipp-a-586469-56986d4c6c from 0 to 1 21m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-586469-7b8b9f445 Created pod: payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc 21m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c Created pod: payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 21m Normal Created pod/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc Container created 21m Normal AddedInterface pod/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc Add eth0 [10.134.1.36/23] from ovn-kubernetes 21m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 Add eth0 [10.134.1.35/23] from ovn-kubernetes 21m Normal Pulled pod/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 21m Normal Created pod/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 Container created 21m Normal Started pod/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 Container started 21m Normal Started pod/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc Container started 21m Normal Killing pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Stopping container payload-processing 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Stopping container istio-proxy 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Stopping container istio-proxy 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-fbcgq Readiness probe failed: HTTP probe failed with statuscode: 503 21m Normal Killing pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Stopping container istio-proxy 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-7wj4l Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-8blfk Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-9h6xp Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-p5gk4 Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-rt957 Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-g2x4p Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Readiness probe error: HTTP probe failed with statuscode: 503... 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-tb8mz Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning Unhealthy pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/e2e-ipp-a-a81979-openshift-default-7c8bcfc9bf-wg5vk Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 4; reason: All metrics below target 21m Normal Killing pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Stopping container istio-proxy 21m Normal SuccessfulDelete replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Deleted pod: maas-default-gateway-openshift-default-5d84bc6b95-hl6bc 21m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled down replica set maas-default-gateway-openshift-default-5d84bc6b95 from 5 to 4 21m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Readiness probe failed: HTTP probe failed with statuscode: 503 21m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-hl6bc Readiness probe error: HTTP probe failed with statuscode: 503... 21m Normal Killing pod/payload-pre-processing-e2e-ipp-b-f9ca72-7c4487d5f-86rtc Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-ipp-b-f9ca72-6bf6ff6f9b-qgtgc Stopping container payload-processing 21m Normal Killing pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Stopping container payload-pre-processing 21m Normal Killing pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Stopping container payload-processing 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w1-599291-68bc448d7-rk4lv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w1-599291-7b7c7fccf9-bqvfh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-ipp-b-f9ca72-openshift-default-5c499594fc-5hwkt Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-n4zh9 Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Killing pod/payload-processing-e2e-ipp-b-403dff-7f4fb76d66-gd69b Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-b-403dff-55687f99-qwrlg Stopping container payload-pre-processing 20m Warning ProbeError pod/e2e-shared-b-w1-599291-openshift-default-69f444c65c-zv8nt Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/payload-processing-e2e-ipp-a-586469-7b8b9f445-scvfc Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-ipp-a-586469-56986d4c6c-4knt8 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Stopping container payload-pre-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w5-974711-6c8dc687c9-bpnhk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w5-974711-5c9c544565-7dvrs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Stopping container payload-processing 20m Normal Killing pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Stopping container istio-proxy 20m Normal Killing pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Stopping container payload-pre-processing 20m Normal Killing pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Stopping container istio-proxy 20m Warning ProbeError pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w5-974711-openshift-default-797ffd4df5-g57p2 Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-a-586469-798774d6c5 Error creating: pods "payload-pre-processing-e2e-ipp-a-586469-798774d6c5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-586469: serviceaccount "payload-processing-e2e-ipp-a-586469" not found 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-a-586469 Scaled up replica set payload-pre-processing-e2e-ipp-a-586469-798774d6c5 from 0 to 1 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-a-586469 Scaled up replica set payload-processing-e2e-ipp-a-586469-7fd596fb78 from 0 to 1 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-a-586469-7fd596fb78 Error creating: pods "payload-processing-e2e-ipp-a-586469-7fd596fb78-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-a-586469: serviceaccount "payload-processing-e2e-ipp-a-586469" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-a-586469-798774d6c5 Created pod: payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal Started pod/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj Container started 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-a-586469-7fd596fb78 Created pod: payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt 20m Normal Created pod/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj Container created 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj Add eth0 [10.134.1.41/23] from ovn-kubernetes 20m Normal Started pod/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt Container started 20m Normal Created pod/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt Container created 20m Normal Pulled pod/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt Add eth0 [10.134.1.40/23] from ovn-kubernetes 20m Warning ProbeError pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-ipp-b-403dff-openshift-default-7656f86cf5-4c46f Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal Pulled pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 20m Normal Started pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Container started 20m Normal Created pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Container created 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Stopping container payload-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Stopping container payload-processing 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w3-33c10a-67fc79744c-pfszb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Stopping container payload-pre-processing 20m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-586469-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-586469-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 20m Normal Killing pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-g4g8r Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Startup probe failed: Get "http://10.134.1.34:15021/healthz/ready": dial tcp 10.134.1.34:15021: connect: connection refused 20m Warning ProbeError pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Startup probe error: Get "http://10.134.1.34:15021/healthz/ready": dial tcp 10.134.1.34:15021: connect: connection refused... 20m Warning ProbeError pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-a-w3-33c10a-openshift-default-b667dc5bc-nvdtn Readiness probe failed: HTTP probe failed with statuscode: 503 20m Normal AddedInterface pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Add eth0 [10.132.1.67/23] from ovn-kubernetes 20m Normal SuccessfulCreate replicaset/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c Created pod: e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj 20m Normal ScalingReplicaSet deployment/e2e-ipp-b-1b75f9-openshift-default Scaled up replica set e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c from 0 to 1 20m Normal AddedLabel gateway/e2e-ipp-b-1b75f9 Added label istio.io/rev=openshift-gateway to gateway e2e-ipp-b-1b75f9 20m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w1-d85d82-678fdc7bff-gbnv8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w3-33c10a-7bd6bddc85-h5w8f Readiness probe failed: timeout: failed to connect service "10.132.0.139:50052" within 1s: context deadline exceeded 20m Warning Unhealthy pod/payload-processing-e2e-worker-w1-d85d82-655545f7d7-q6jbs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Stopping container payload-processing 20m Normal Started pod/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh Container started 20m Normal Started pod/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 Container started 20m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 Add eth0 [10.134.1.46/23] from ovn-kubernetes 20m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1b75f9 Scaled up replica set payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f from 0 to 1 20m Normal Created pod/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 Container created 20m Normal Created pod/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh Container created 20m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1b75f9 Scaled up replica set payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c from 0 to 1 20m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh Add eth0 [10.134.1.45/23] from ovn-kubernetes 20m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c Error creating: pods "payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1b75f9: serviceaccount "payload-processing-e2e-ipp-b-1b75f9" not found 20m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c Created pod: payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh 20m Normal Pulled pod/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 20m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f Created pod: payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 20m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f Error creating: pods "payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1b75f9: serviceaccount "payload-processing-e2e-ipp-b-1b75f9" not found 20m Warning Unhealthy pod/payload-processing-e2e-shared-b-w5-14ba5a-845bd58c69-zkbf8 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/payload-processing-e2e-shared-a-w6-15dc2f-5c94c8878c-24mmr Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Stopping container istio-proxy 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w5-14ba5a-6dc47df64c-sf74f Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Normal Killing pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-2gpc4 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-a-w6-15dc2f-openshift-default-b5c8767cc-hv9c9 Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-hvcs2 Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w6-15dc2f-fdc4cf6c-zgv5v Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 20m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-cr8gx Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-6g55f Readiness probe error: HTTP probe failed with statuscode: 503... 20m Warning Unhealthy pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-shared-b-w5-14ba5a-openshift-default-66ccc567f-pppvh Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Stopping container istio-proxy 20m Normal Killing pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Stopping container istio-proxy 20m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Readiness probe failed: HTTP probe failed with statuscode: 503 20m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-w7bgs Readiness probe error: HTTP probe failed with statuscode: 503... 20m Normal Killing pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Stopping container payload-processing 20m Warning Unhealthy pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Readiness probe failed: Get "http://10.134.0.103:15021/healthz/ready": dial tcp 10.134.0.103:15021: connect: connection refused 20m Warning ProbeError pod/e2e-worker-w1-d85d82-openshift-default-7b77944d8f-4q6cd Readiness probe error: Get "http://10.134.0.103:15021/healthz/ready": dial tcp 10.134.0.103:15021: connect: connection refused... 20m Normal Killing pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Stopping container payload-pre-processing 20m Normal Killing pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Stopping container payload-processing 20m Normal Killing pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Stopping container payload-pre-processing 20m Normal Killing pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-8zd7n Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-74ns6 Stopping container istio-proxy 20m Normal Killing pod/e2e-shared-b-w3-abdcb1-openshift-default-6b5875986f-kq25j Stopping container istio-proxy 19m Normal Killing pod/payload-processing-e2e-ipp-b-1b75f9-fbd9bf65f-f5d84 Stopping container payload-processing 19m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1b75f9-7cb5494d4c-jnqnh Stopping container payload-pre-processing 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w3-abdcb1-99bb4999f-ftjr9 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w3-abdcb1-858c897c99-r6fx5 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Stopping container payload-pre-processing 19m Normal SuccessfulCreate replicaset/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d Created pod: payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz 19m Normal Killing pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Stopping container istio-proxy 19m Normal AddedInterface pod/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv Add eth0 [10.134.1.50/23] from ovn-kubernetes 19m Normal Pulled pod/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Normal Pulled pod/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz Container image "quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable" already present on machine and can be accessed by the pod 19m Warning FailedCreate replicaset/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c Error creating: pods "payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1b75f9: serviceaccount "payload-processing-e2e-ipp-b-1b75f9" not found 19m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c Created pod: payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv 19m Normal Killing pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Stopping container payload-processing 19m Normal AddedInterface pod/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz Add eth0 [10.132.1.71/23] from ovn-kubernetes 19m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ipp-b-1b75f9 Scaled up replica set payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c from 0 to 1 19m Normal Started pod/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz Container started 19m Normal Created pod/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz Container created 19m Normal Killing pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-fkrpb Stopping container istio-proxy 19m Warning FailedCreate replicaset/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d Error creating: pods "payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ipp-b-1b75f9: serviceaccount "payload-processing-e2e-ipp-b-1b75f9" not found 19m Normal ScalingReplicaSet deployment/payload-processing-e2e-ipp-b-1b75f9 Scaled up replica set payload-processing-e2e-ipp-b-1b75f9-57c6b6576d from 0 to 1 19m Normal Created pod/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv Container created 19m Warning ProbeError pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-shared-b-w6-5e8f29-openshift-default-745c6666c9-flxpv Readiness probe failed: HTTP probe failed with statuscode: 503 19m Normal Started pod/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv Container started 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-a-586469-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-a-586469-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning Unhealthy pod/payload-processing-e2e-shared-b-w6-5e8f29-94bb49845-c6tvg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w6-5e8f29-5788d794dc-xtw28 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w5-6784d6-d5845499-shd7t Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Stopping container payload-pre-processing 19m Warning Unhealthy pod/payload-processing-e2e-worker-w5-6784d6-6dfd744559-gfw5q Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Stopping container payload-processing 19m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-bgc5j Stopping container istio-proxy 19m Normal Pulled pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 19m Normal Killing pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Stopping container istio-proxy 19m Normal Created pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Container created 19m Normal Started pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Container started 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w3-c82140-68b5fb4d55-8w8nb Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning ProbeError pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/e2e-worker-w5-6784d6-openshift-default-5789cc64d5-nvnv5 Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning Unhealthy pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Startup probe failed: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused 19m Warning ProbeError pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Startup probe error: Get "http://10.132.1.67:15021/healthz/ready": dial tcp 10.132.1.67:15021: connect: connection refused... 19m Warning Unhealthy pod/payload-processing-e2e-worker-w3-c82140-575b9b7477-5qnmw Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/payload-pre-processing-e2e-ipp-a-586469-798774d6c5-pxksj Stopping container payload-pre-processing 19m Normal Killing pod/payload-processing-e2e-ipp-a-586469-7fd596fb78-28rgt Stopping container payload-processing 19m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-l8pj5 Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Stopping container istio-proxy 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1b75f9-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1b75f9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 19m Warning Unhealthy pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w3-c82140-openshift-default-6c6b7c6899-gpdxj Readiness probe error: HTTP probe failed with statuscode: 503... 19m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w6-aabeef-585b7b8c4c-gbl4w Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Warning Unhealthy pod/payload-processing-e2e-worker-w6-aabeef-7fbf76c96c-cx8wp Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 19m Normal Killing pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Stopping container istio-proxy 19m Warning Unhealthy pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-ipp-a-586469-openshift-default-6cb999f45f-7q54m Readiness probe error: HTTP probe failed with statuscode: 503... 19m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-f8ccq Stopping container istio-proxy 19m Normal Killing pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Stopping container istio-proxy 19m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ipp-b-1b75f9-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ipp-b-1b75f9-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 19m Warning Unhealthy pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Readiness probe failed: HTTP probe failed with statuscode: 503 19m Warning ProbeError pod/e2e-worker-w6-aabeef-openshift-default-6b8d7f885c-zrmtj Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/payload-pre-processing-e2e-ipp-b-1b75f9-646d8c996c-rscwv Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-ipp-b-1b75f9-57c6b6576d-p2kzz Stopping container payload-processing 18m Normal Killing pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Stopping container istio-proxy 18m Warning ProbeError pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Readiness probe error: HTTP probe failed with statuscode: 503... 18m Warning Unhealthy pod/e2e-ipp-b-1b75f9-openshift-default-656d6dfd7c-zl9kj Readiness probe failed: HTTP probe failed with statuscode: 503 18m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-gk567 Stopping container discovery 18m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 3 to 2 18m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-gk567 18m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 2; reason: All metrics below target 18m Normal Killing pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Stopping container payload-processing 18m Normal Killing pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Stopping container payload-pre-processing 18m Normal Killing pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-s6znk Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-vh9q9 Stopping container istio-proxy 18m Warning Unhealthy pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Readiness probe failed: HTTP probe failed with statuscode: 503 18m Warning ProbeError pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Readiness probe error: HTTP probe failed with statuscode: 503... 18m Normal Killing pod/e2e-shared-a-w4-a39e7e-openshift-default-58476ffb8c-svkjh Stopping container istio-proxy 18m Warning Unhealthy pod/payload-processing-e2e-shared-a-w4-a39e7e-64f4c7ffdf-zfkfk Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Stopping container payload-pre-processing 18m Normal Killing pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Stopping container payload-processing 18m Warning Unhealthy pod/payload-pre-processing-e2e-shared-a-w4-a39e7e-547df6896c-n9726 Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 18m Normal Killing pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-7n68s Stopping container istio-proxy 18m Normal Killing pod/e2e-shared-b-w4-35d13e-openshift-default-cb5cd6b99-wcr5z Stopping container istio-proxy 17m Warning Unhealthy pod/payload-pre-processing-e2e-shared-b-w4-35d13e-76bf444c-bsqvs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-processing-e2e-shared-b-w4-35d13e-5857c666d9-fs4kh Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Stopping container payload-pre-processing 17m Normal Killing pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Stopping container payload-processing 17m Warning Unhealthy pod/payload-processing-e2e-worker-w4-8b2f0a-544f99cf49-r4qxs Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Warning Unhealthy pod/payload-pre-processing-e2e-worker-w4-8b2f0a-758b55ccf9-kfzhg Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 17m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Stopping container istio-proxy 17m Normal Killing pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Stopping container istio-proxy 17m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Readiness probe failed: HTTP probe failed with statuscode: 503 17m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-rbb6h Readiness probe error: HTTP probe failed with statuscode: 503... 17m Warning Unhealthy pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Readiness probe failed: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused 17m Warning ProbeError pod/e2e-worker-w4-8b2f0a-openshift-default-64797b57bc-l5cvv Readiness probe error: Get "http://10.133.0.81:15021/healthz/ready": dial tcp 10.133.0.81:15021: connect: connection refused... 17m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5d84bc6b95 from 4 to 8 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-tvm4p 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-5z724 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-76bb5 17m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-7sz9s 17m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 8; reason: cpu resource utilization (percentage of request) above target 17m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Container created 17m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Container created 17m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Container started 17m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Container started 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Add eth0 [10.132.1.79/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Add eth0 [10.132.1.78/23] from ovn-kubernetes 17m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Container created 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Add eth0 [10.134.1.58/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Container started 17m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Container started 17m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Add eth0 [10.133.0.198/23] from ovn-kubernetes 17m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 17m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Container created 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Startup probe failed: Get "http://10.132.1.78:15021/healthz/ready": dial tcp 10.132.1.78:15021: connect: connection refused 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Startup probe failed: Get "http://10.132.1.79:15021/healthz/ready": dial tcp 10.132.1.79:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": dial tcp 10.133.0.198:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Startup probe error: Get "http://10.132.1.79:15021/healthz/ready": dial tcp 10.132.1.79:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Startup probe error: Get "http://10.132.1.78:15021/healthz/ready": dial tcp 10.132.1.78:15021: connect: connection refused... 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Startup probe error: Get "http://10.134.1.58:15021/healthz/ready": dial tcp 10.134.1.58:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Startup probe failed: Get "http://10.134.1.58:15021/healthz/ready": dial tcp 10.134.1.58:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Startup probe error: Get "http://10.134.1.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-7sz9s Startup probe failed: Get "http://10.134.1.58:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Startup probe error: Get "http://10.132.1.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-76bb5 Startup probe failed: Get "http://10.132.1.79:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Startup probe failed: Get "http://10.133.0.198:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-5z724 Startup probe error: Get "http://10.133.0.198:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Startup probe failed: Get "http://10.132.1.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Startup probe error: Get "http://10.132.1.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 16m Normal ScalingReplicaSet deployment/maas-default-gateway-openshift-default Scaled up replica set maas-default-gateway-openshift-default-5d84bc6b95 from 8 to 10 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 16m Normal SuccessfulRescale horizontalpodautoscaler/maas-default-gateway-openshift-default New size: 10; reason: All metrics below target 16m Normal SuccessfulCreate replicaset/maas-default-gateway-openshift-default-5d84bc6b95 Created pod: maas-default-gateway-openshift-default-5d84bc6b95-gwl2w 16m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Container started 16m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Container created 16m Normal Pulled pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Add eth0 [10.133.0.199/23] from ovn-kubernetes 16m Normal Created pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Container created 16m Normal Started pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Container started 16m Normal AddedInterface pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Add eth0 [10.132.1.81/23] from ovn-kubernetes 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Startup probe error: Get "http://10.132.1.81:15021/healthz/ready": dial tcp 10.132.1.81:15021: connect: connection refused... 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Startup probe failed: Get "http://10.132.1.81:15021/healthz/ready": dial tcp 10.132.1.81:15021: connect: connection refused 16m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Startup probe failed: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused 16m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Startup probe error: Get "http://10.133.0.199:15021/healthz/ready": dial tcp 10.133.0.199:15021: connect: connection refused... 14m Normal SuccessfulRescale horizontalpodautoscaler/istiod-openshift-gateway New size: 1; reason: All metrics below target 14m Normal ScalingReplicaSet deployment/istiod-openshift-gateway Scaled down replica set istiod-openshift-gateway-7ddd9cfdc8 from 2 to 1 14m Normal SuccessfulDelete replicaset/istiod-openshift-gateway-7ddd9cfdc8 Deleted pod: istiod-openshift-gateway-7ddd9cfdc8-thzmx 14m Normal Killing pod/istiod-openshift-gateway-7ddd9cfdc8-thzmx Stopping container discovery 12m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Readiness probe error: Get "http://10.132.1.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Readiness probe failed: Get "http://10.133.0.199:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-gwl2w Readiness probe error: Get "http://10.133.0.199:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-qhgv4 Readiness probe failed: Get "http://10.132.1.81:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Readiness probe failed: Get "http://10.132.1.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 12m Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-tvm4p Readiness probe error: Get "http://10.132.1.78:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 12m Normal SuccessfulCreate replicaset/e2e-worker-single-main-c4292c-openshift-default-c85866885 Created pod: e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 12m Normal ScalingReplicaSet deployment/e2e-worker-single-main-c4292c-openshift-default Scaled up replica set e2e-worker-single-main-c4292c-openshift-default-c85866885 from 0 to 1 12m Normal AddedLabel gateway/e2e-worker-single-main-c4292c Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-c4292c 12m Normal AddedInterface pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Add eth0 [10.134.1.74/23] from ovn-kubernetes 12m Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-c4292c-c45b8d99c Created pod: payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv 12m Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-c4292c-c45b8d99c Error creating: pods "payload-processing-e2e-worker-single-main-c4292c-c45b8d99c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-c4292c: serviceaccount "payload-processing-e2e-worker-single-main-c4292c" not found 12m Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-c4292c-5486c5797f Created pod: payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf 12m Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-c4292c-5486c5797f Error creating: pods "payload-pre-processing-e2e-worker-single-main-c4292c-5486c5797f-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-c4292c: serviceaccount "payload-processing-e2e-worker-single-main-c4292c" not found 12m Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-c4292c Scaled up replica set payload-pre-processing-e2e-worker-single-main-c4292c-5486c5797f from 0 to 1 12m Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-c4292c Scaled up replica set payload-processing-e2e-worker-single-main-c4292c-c45b8d99c from 0 to 1 11m Normal Pulled pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal Created pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Container created 11m Normal Started pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Container started 11m Normal Created pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Container created 11m Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Add eth0 [10.134.1.76/23] from ovn-kubernetes 11m Normal Started pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Container started 11m Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 11m Normal AddedInterface pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Add eth0 [10.134.1.77/23] from ovn-kubernetes 11m Warning Unhealthy pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Readiness probe failed: Get "http://10.134.1.74:15021/healthz/ready": read tcp 10.134.0.2:59886->10.134.1.74:15021: read: connection reset by peer 11m Warning ProbeError pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Readiness probe error: Get "http://10.134.1.74:15021/healthz/ready": read tcp 10.134.0.2:59886->10.134.1.74:15021: read: connection reset by peer... 11m Normal Started pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Container started 11m Normal Created pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Container created 11m Normal Pulled pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 11m Warning ProbeError pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Startup probe error: Get "http://10.134.1.74:15021/healthz/ready": dial tcp 10.134.1.74:15021: connect: connection refused... 11m Warning Unhealthy pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Startup probe failed: Get "http://10.134.1.74:15021/healthz/ready": dial tcp 10.134.1.74:15021: connect: connection refused 11m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-c4292c-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 11m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-c4292c-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 10m Normal Killing pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Stopping container payload-processing 10m Normal Killing pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Stopping container payload-pre-processing 10m Normal Killing pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Stopping container istio-proxy 10m Warning Unhealthy pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Readiness probe failed: HTTP probe failed with statuscode: 503 10m Warning ProbeError pod/e2e-worker-single-main-c4292c-openshift-default-c85866885-pk4x4 Readiness probe error: HTTP probe failed with statuscode: 503... 10m Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-c4292c-5486c2qzvf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 10m Warning Unhealthy pod/payload-processing-e2e-worker-single-main-c4292c-c45b8d99cmjnzv Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 8m18s Normal AddedInterface pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Add eth0 [10.134.1.83/23] from ovn-kubernetes 8m18s Normal SuccessfulCreate replicaset/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5df Created pod: e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 8m18s Normal ScalingReplicaSet deployment/e2e-worker-single-main-a9dd7f-openshift-default Scaled up replica set e2e-worker-single-main-a9dd7f-openshift-default-7667cff5df from 0 to 1 8m18s Normal AddedLabel gateway/e2e-worker-single-main-a9dd7f Added label istio.io/rev=openshift-gateway to gateway e2e-worker-single-main-a9dd7f 8m14s Warning FailedCreate replicaset/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfd5 Error creating: pods "payload-processing-e2e-worker-single-main-a9dd7f-86f567bfd5-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a9dd7f: serviceaccount "payload-processing-e2e-worker-single-main-a9dd7f" not found 8m14s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-worker-single-main-a9dd7f Scaled up replica set payload-pre-processing-e2e-worker-single-main-a9dd7f-587bf54659 from 0 to 1 8m14s Warning FailedCreate replicaset/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bf54659 Error creating: pods "payload-pre-processing-e2e-worker-single-main-a9dd7f-587bf54659-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-worker-single-main-a9dd7f: serviceaccount "payload-processing-e2e-worker-single-main-a9dd7f" not found 8m14s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bf54659 Created pod: payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf 8m14s Normal ScalingReplicaSet deployment/payload-processing-e2e-worker-single-main-a9dd7f Scaled up replica set payload-processing-e2e-worker-single-main-a9dd7f-86f567bfd5 from 0 to 1 8m14s Normal SuccessfulCreate replicaset/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfd5 Created pod: payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s 8m13s Normal AddedInterface pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Add eth0 [10.134.1.85/23] from ovn-kubernetes 8m13s Normal Created pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Container created 8m13s Normal Created pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Container created 8m13s Normal Pulled pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m13s Normal Started pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Container started 8m13s Normal AddedInterface pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Add eth0 [10.134.1.86/23] from ovn-kubernetes 8m13s Normal Started pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Container started 8m13s Normal Pulled pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 8m1s Normal Created pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Container created 8m1s Warning Unhealthy pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Readiness probe failed: Get "http://10.134.1.83:15021/healthz/ready": dial tcp 10.134.1.83:15021: connect: connection refused 8m1s Warning ProbeError pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Readiness probe error: Get "http://10.134.1.83:15021/healthz/ready": dial tcp 10.134.1.83:15021: connect: connection refused... 8m1s Normal Started pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Container started 8m1s Normal Pulled pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 7m56s Warning Unhealthy pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Startup probe failed: Get "http://10.134.1.83:15021/healthz/ready": dial tcp 10.134.1.83:15021: connect: connection refused 7m56s Warning ProbeError pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Startup probe error: Get "http://10.134.1.83:15021/healthz/ready": dial tcp 10.134.1.83:15021: connect: connection refused... 7m48s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a9dd7f-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m48s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a9dd7f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 7m33s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-worker-single-main-a9dd7f-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m33s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-worker-single-main-a9dd7f-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 7m7s Normal Killing pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Stopping container payload-pre-processing 7m7s Normal Killing pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Stopping container payload-processing 6m58s Normal Killing pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Stopping container istio-proxy 6m55s Warning Unhealthy pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Readiness probe failed: HTTP probe failed with statuscode: 503 6m55s Warning ProbeError pod/e2e-worker-single-main-a9dd7f-openshift-default-7667cff5dfw4mw5 Readiness probe error: HTTP probe failed with statuscode: 503... 6m51s Warning Unhealthy pod/payload-processing-e2e-worker-single-main-a9dd7f-86f567bfdgcd6s Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 6m51s Warning Unhealthy pod/payload-pre-processing-e2e-worker-single-main-a9dd7f-587bfqsvhf Readiness probe failed: service unhealthy (responded with "NOT_SERVING") 5m48s Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Readiness probe failed: Get "http://10.134.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m48s Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-n56bm Readiness probe error: Get "http://10.134.0.31:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m48s Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Readiness probe failed: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 5m48s Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-hdtr7 Readiness probe error: Get "http://10.132.0.36:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 5m23s Normal AddedLabel gateway/e2e-ait-a1008181 Added label istio.io/rev=openshift-gateway to gateway e2e-ait-a1008181 5m23s Normal ScalingReplicaSet deployment/e2e-ait-a1008181-openshift-default Scaled up replica set e2e-ait-a1008181-openshift-default-57476b49cc from 0 to 1 5m23s Normal SuccessfulCreate replicaset/e2e-ait-a1008181-openshift-default-57476b49cc Created pod: e2e-ait-a1008181-openshift-default-57476b49cc-b4htz 5m22s Normal AddedInterface pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Add eth0 [10.134.1.96/23] from ovn-kubernetes 5m18s Normal ScalingReplicaSet deployment/payload-processing-e2e-ait-a1008181 Scaled up replica set payload-processing-e2e-ait-a1008181-55d6d486cf from 0 to 1 5m18s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-ait-a1008181-77b484dbbc Created pod: payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 5m18s Warning FailedCreate replicaset/payload-processing-e2e-ait-a1008181-55d6d486cf Error creating: pods "payload-processing-e2e-ait-a1008181-55d6d486cf-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a1008181: serviceaccount "payload-processing-e2e-ait-a1008181" not found 5m18s Warning FailedCreate replicaset/payload-pre-processing-e2e-ait-a1008181-77b484dbbc Error creating: pods "payload-pre-processing-e2e-ait-a1008181-77b484dbbc-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-ait-a1008181: serviceaccount "payload-processing-e2e-ait-a1008181" not found 5m18s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-ait-a1008181 Scaled up replica set payload-pre-processing-e2e-ait-a1008181-77b484dbbc from 0 to 1 5m18s Normal SuccessfulCreate replicaset/payload-processing-e2e-ait-a1008181-55d6d486cf Created pod: payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs 5m17s Normal AddedInterface pod/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs Add eth0 [10.134.1.99/23] from ovn-kubernetes 5m17s Normal Created pod/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 Container created 5m17s Normal Started pod/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 Container started 5m17s Normal Pulled pod/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m17s Normal AddedInterface pod/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 Add eth0 [10.134.1.98/23] from ovn-kubernetes 5m17s Normal Pulled pod/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 5m17s Normal Created pod/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs Container created 5m17s Normal Started pod/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs Container started 5m8s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a1008181-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m8s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a1008181-openshift-default failed to get cpu utilization: did not receive metrics for targeted pods (pods might be unready) 5m6s Normal Started pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Container started 5m6s Normal Created pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Container created 5m6s Normal Pulled pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 5m Warning Unhealthy pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Startup probe failed: Get "http://10.134.1.96:15021/healthz/ready": dial tcp 10.134.1.96:15021: connect: connection refused 5m Warning ProbeError pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Startup probe error: Get "http://10.134.1.96:15021/healthz/ready": dial tcp 10.134.1.96:15021: connect: connection refused... 4m53s Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-ait-a1008181-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m53s Normal Killing pod/payload-pre-processing-e2e-ait-a1008181-77b484dbbc-4vdt6 Stopping container payload-pre-processing 4m53s Normal Killing pod/payload-processing-e2e-ait-a1008181-55d6d486cf-wpkvs Stopping container payload-processing 4m53s Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-ait-a1008181-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m47s Normal Killing pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Stopping container istio-proxy 4m45s Normal ScalingReplicaSet deployment/e2e-authcfg-http-c67e06-openshift-default Scaled up replica set e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9 from 0 to 1 4m45s Normal SuccessfulCreate replicaset/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9 Created pod: e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 4m45s Normal AddedLabel gateway/e2e-authcfg-http-c67e06 Added label istio.io/rev=openshift-gateway to gateway e2e-authcfg-http-c67e06 4m45s Normal AddedInterface pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Add eth0 [10.134.1.100/23] from ovn-kubernetes 4m44s Warning Unhealthy pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Readiness probe failed: HTTP probe failed with statuscode: 503 4m44s Warning ProbeError pod/e2e-ait-a1008181-openshift-default-57476b49cc-b4htz Readiness probe error: HTTP probe failed with statuscode: 503... 4m40s Warning FailedCreate replicaset/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df Error creating: pods "payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-c67e06: serviceaccount "payload-processing-e2e-authcfg-http-c67e06" not found 4m40s Normal ScalingReplicaSet deployment/payload-processing-e2e-authcfg-http-c67e06 Scaled up replica set payload-processing-e2e-authcfg-http-c67e06-bcf9db5df from 0 to 1 4m40s Normal SuccessfulCreate replicaset/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c Created pod: payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 4m40s Normal ScalingReplicaSet deployment/payload-pre-processing-e2e-authcfg-http-c67e06 Scaled up replica set payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c from 0 to 1 4m40s Warning FailedCreate replicaset/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c Error creating: pods "payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-" is forbidden: error looking up service account openshift-ingress/payload-processing-e2e-authcfg-http-c67e06: serviceaccount "payload-processing-e2e-authcfg-http-c67e06" not found 4m40s Normal SuccessfulCreate replicaset/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df Created pod: payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj 4m39s Normal Started pod/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj Container started 4m39s Normal Created pod/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj Container created 4m39s Normal AddedInterface pod/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj Add eth0 [10.134.1.103/23] from ovn-kubernetes 4m39s Normal AddedInterface pod/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 Add eth0 [10.134.1.102/23] from ovn-kubernetes 4m39s Normal Pulled pod/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m39s Normal Pulled pod/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj Container image "quay.io/opendatahub/odh-praxis-extproc:odh-stable" already present on machine and can be accessed by the pod 4m39s Normal Started pod/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 Container started 4m39s Normal Created pod/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 Container created 4m27s Warning Unhealthy pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Readiness probe failed: Get "http://10.134.1.100:15021/healthz/ready": read tcp 10.134.0.2:50470->10.134.1.100:15021: read: connection reset by peer 4m27s Warning ProbeError pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Readiness probe error: Get "http://10.134.1.100:15021/healthz/ready": read tcp 10.134.0.2:50470->10.134.1.100:15021: read: connection reset by peer... 4m26s Normal Created pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Container created 4m26s Normal Started pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Container started 4m26s Normal Pulled pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Container image "registry.redhat.io/openshift-service-mesh/istio-proxyv2-rhel9@sha256:1fc05d17a76d31503f68c5bde3057750a2c575981535756112c06589f88c8c55" already present on machine and can be accessed by the pod 4m21s Warning Unhealthy pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Startup probe failed: Get "http://10.134.1.100:15021/healthz/ready": dial tcp 10.134.1.100:15021: connect: connection refused 4m21s Warning ProbeError pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Startup probe error: Get "http://10.134.1.100:15021/healthz/ready": dial tcp 10.134.1.100:15021: connect: connection refused... 4m Warning FailedComputeMetricsReplicas horizontalpodautoscaler/e2e-authcfg-http-c67e06-openshift-default invalid metrics (1 invalid out of 1), first error is: failed to get cpu resource metric value: failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 4m Warning FailedGetResourceMetric horizontalpodautoscaler/e2e-authcfg-http-c67e06-openshift-default failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API 2m56s Normal Killing pod/payload-pre-processing-e2e-authcfg-http-c67e06-687bdc754c-q7sm8 Stopping container payload-pre-processing 2m56s Normal Killing pod/payload-processing-e2e-authcfg-http-c67e06-bcf9db5df-584lj Stopping container payload-processing 2m42s Normal Killing pod/e2e-authcfg-http-c67e06-openshift-default-565cd9b9d9-mnjj7 Stopping container istio-proxy 2m32s Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Readiness probe failed: Get "http://10.133.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m32s Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-2fpt9 Readiness probe error: Get "http://10.133.0.32:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)... 2m15s Warning Unhealthy pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Readiness probe failed: Get "http://10.134.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers) 2m15s Warning ProbeError pod/maas-default-gateway-openshift-default-5d84bc6b95-66bvk Readiness probe error: Get "http://10.134.0.30:15021/healthz/ready": context deadline exceeded (Client.Timeout exceeded while awaiting headers)...